-- Logs begin at Wed 2022-08-31 10:03:34 UTC, end at Wed 2022-08-31 11:57:41 UTC. -- Aug 31 10:43:15.364272 dv-cinder-846173-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 31 10:43:15.373159 dv-cinder-846173-5-iscsi devstack@keystone.service[82076]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 31 10:43:15.375374 dv-cinder-846173-5-iscsi devstack@keystone.service[82076]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 31 10:43:15 2022] *** Aug 31 10:43:15.375374 dv-cinder-846173-5-iscsi devstack@keystone.service[82076]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 31 10:43:15.375374 dv-cinder-846173-5-iscsi devstack@keystone.service[82076]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 31 10:43:15.375374 dv-cinder-846173-5-iscsi devstack@keystone.service[82076]: nodename: dv-cinder-846173-5-iscsi Aug 31 10:43:15.375374 dv-cinder-846173-5-iscsi devstack@keystone.service[82076]: machine: x86_64 Aug 31 10:43:15.375374 dv-cinder-846173-5-iscsi devstack@keystone.service[82076]: clock source: unix Aug 31 10:43:15.375374 dv-cinder-846173-5-iscsi devstack@keystone.service[82076]: pcre jit disabled Aug 31 10:43:15.375374 dv-cinder-846173-5-iscsi devstack@keystone.service[82076]: detected number of CPU cores: 4 Aug 31 10:43:15.375374 dv-cinder-846173-5-iscsi devstack@keystone.service[82076]: current working directory: / Aug 31 10:43:15.375374 dv-cinder-846173-5-iscsi devstack@keystone.service[82076]: detected binary path: /usr/bin/uwsgi-core Aug 31 10:43:15.375374 dv-cinder-846173-5-iscsi devstack@keystone.service[82076]: your processes number limit is 47680 Aug 31 10:43:15.376105 dv-cinder-846173-5-iscsi devstack@keystone.service[82076]: your memory page size is 4096 bytes Aug 31 10:43:15.376105 dv-cinder-846173-5-iscsi devstack@keystone.service[82076]: detected max file descriptor number: 2048 Aug 31 10:43:15.376105 dv-cinder-846173-5-iscsi devstack@keystone.service[82076]: lock engine: pthread robust mutexes Aug 31 10:43:15.376105 dv-cinder-846173-5-iscsi devstack@keystone.service[82076]: thunder lock: enabled Aug 31 10:43:15.376105 dv-cinder-846173-5-iscsi devstack@keystone.service[82076]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 31 10:43:15.376105 dv-cinder-846173-5-iscsi devstack@keystone.service[82076]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 31 10:43:15.416243 dv-cinder-846173-5-iscsi devstack@keystone.service[82076]: Python main interpreter initialized at 0x55775268f9c0 Aug 31 10:43:15.416243 dv-cinder-846173-5-iscsi devstack@keystone.service[82076]: python threads support enabled Aug 31 10:43:15.416243 dv-cinder-846173-5-iscsi devstack@keystone.service[82076]: your server socket listen backlog is limited to 100 connections Aug 31 10:43:15.416243 dv-cinder-846173-5-iscsi devstack@keystone.service[82076]: your mercy for graceful operations on workers is 90 seconds Aug 31 10:43:15.416632 dv-cinder-846173-5-iscsi devstack@keystone.service[82076]: mapped 403077 bytes (393 KB) for 2 cores Aug 31 10:43:15.416710 dv-cinder-846173-5-iscsi devstack@keystone.service[82076]: *** Operational MODE: preforking *** Aug 31 10:43:15.416967 dv-cinder-846173-5-iscsi devstack@keystone.service[82076]: *** uWSGI is running in multiple interpreter mode *** Aug 31 10:43:15.416967 dv-cinder-846173-5-iscsi devstack@keystone.service[82076]: spawned uWSGI master process (pid: 82076) Aug 31 10:43:15.417186 dv-cinder-846173-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 31 10:43:15.418011 dv-cinder-846173-5-iscsi devstack@keystone.service[82076]: spawned uWSGI worker 1 (pid: 82088, cores: 1) Aug 31 10:43:15.418109 dv-cinder-846173-5-iscsi devstack@keystone.service[82076]: spawned uWSGI worker 2 (pid: 82089, cores: 1) Aug 31 10:43:15.420054 dv-cinder-846173-5-iscsi devstack@keystone.service[82076]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 31 10:43:19.807167 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 10:43:19.807167 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: from cryptography.utils import int_from_bytes Aug 31 10:43:19.808011 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 10:43:19.808011 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: from cryptography.utils import int_from_bytes Aug 31 10:43:19.809294 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 10:43:19.809294 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: from cryptography.utils import int_from_bytes Aug 31 10:43:19.809775 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 10:43:19.809775 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: from cryptography.utils import int_from_bytes Aug 31 10:43:19.862017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 10:43:19.862201 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 10:43:19.862371 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] command line args: [] {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 10:43:19.862658 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 10:43:19.862834 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ================================================================================ {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 10:43:19.862918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 10:43:19.863065 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 10:43:19.863155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] admin_token = **** {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.863256 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] command line args: [] {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 10:43:19.863338 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] config_dir = [] {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.863486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.863563 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 10:43:19.863666 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] config_source = [] {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.863751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ================================================================================ {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 10:43:19.863829 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] debug = True {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.863982 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] admin_token = **** {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.864077 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.864264 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] config_dir = [] {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.864396 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.864472 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.864547 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] config_source = [] {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.864604 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] insecure_debug = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.864728 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] debug = True {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.864845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.864952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.865082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.865184 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.865278 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] list_limit = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.865379 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] insecure_debug = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.865480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] log_config_append = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.865586 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.865685 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.865778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] log_dir = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.865876 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.866121 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] log_file = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.866234 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] list_limit = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.866329 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.866467 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] log_config_append = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.866467 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.866680 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.866755 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] log_dir = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.866825 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.866932 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] log_file = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.867016 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.867120 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.867230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.867305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.867413 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.867497 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.867617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.867698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.867829 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.867908 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.867980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.868096 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.868169 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.868288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.868359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.868436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.868581 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.868664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.868825 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.868919 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.869049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.869136 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] notification_format = cadf {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.869278 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.869354 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.869520 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.869761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] notification_format = cadf {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.869936 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.870052 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] publish_errors = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.870163 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.870264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.870384 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.870529 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.870649 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.870767 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] publish_errors = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.870890 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.871009 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.871123 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.871233 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.871364 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] standard_threads = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.871485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.871599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] strict_password_check = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.871715 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.871833 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.871947 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.872058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] use_eventlog = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.872422 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] standard_threads = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.872541 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] use_journal = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.872660 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] strict_password_check = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.872778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] use_json = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.872778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] use_stderr = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.872994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.873183 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] use_syslog = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.873299 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] use_eventlog = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.873299 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] use_journal = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.873508 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] watch_log_file = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.873626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] use_json = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.873745 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.873869 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] use_stderr = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.873983 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.874094 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] use_syslog = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.874094 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] watch_log_file = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:43:19.874307 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.874307 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.879140 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.879260 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.879340 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.879414 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.879483 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.879605 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.879691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.879762 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.879830 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.879944 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.880023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.880023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.880166 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.880230 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.880344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.880417 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.880492 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.880567 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.880636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.880699 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.880781 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.880851 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.880925 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.881008 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.881084 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.881159 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.881229 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.881302 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.881372 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.881443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.881560 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.881670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.881782 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.881861 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.881937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.882086 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.882159 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.882159 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.882288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.882357 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.882493 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.882582 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.882700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.882774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.882843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.882905 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.883018 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.883100 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.883171 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.883235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.883347 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.883458 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.883538 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.883663 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.883739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.883808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.883875 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.883989 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.884084 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.884170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.884247 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.884346 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.884445 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.884539 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.884637 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.884739 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.884840 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.884934 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.885034 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.885133 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.885227 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.885340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.885435 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.885528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.885626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.885757 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.885865 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.885962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.886056 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.886133 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.886255 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.886331 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.886418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.886482 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.886598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.886672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.886742 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.886859 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.886944 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.887005 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.887005 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.887114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.887225 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.887355 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.887433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.887548 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.887635 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.887744 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.887827 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.887930 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.888007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.888066 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.888171 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.888249 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.888363 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.888444 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.888558 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.888700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.888793 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.888911 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.889008 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.889086 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.889192 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.889277 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.889354 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.889420 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.889521 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.889610 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.889683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.889742 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.889859 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.889962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.890040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.890104 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.890212 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.890357 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.890477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.890548 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.890548 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.890659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.890717 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.890773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.890900 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.890966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.891025 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.891081 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.891081 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.891188 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.891301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.891301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.891482 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.891564 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.enabled = True {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.891564 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.891739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.891819 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.891890 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.891960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.892017 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.892132 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.892251 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.892335 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.892416 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.892483 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.892600 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.enabled = True {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.892675 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.892740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.892854 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.892936 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.893045 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.893116 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.893280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.893366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.893547 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.893722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.893831 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.893895 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.894040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.894177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.894260 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.894479 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.894601 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.894728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.894848 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.894976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.894976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.895218 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.895301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.895378 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.895446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.895504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.895589 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.895772 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.895904 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.895986 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.896144 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.896264 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.896385 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.896518 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.896654 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.896783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.896907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.897033 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.897158 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.897292 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.897415 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.897541 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.897682 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.897813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.897938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.898067 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.898194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.898315 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.898488 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.898617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.898737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] auth.external = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.898880 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.898999 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] auth.mapped = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.899123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.899241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.899405 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.899538 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.899660 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.899787 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] auth.password = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.899921 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.900043 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] auth.token = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.900131 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.900246 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.900370 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] auth.external = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.900495 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] catalog.caching = True {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.900625 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] auth.mapped = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.900746 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.900866 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.901000 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.901136 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.901256 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.901389 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] auth.password = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.901512 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.901633 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] auth.token = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.901759 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.901879 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.901960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] credential.caching = True {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.902128 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] catalog.caching = True {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.902259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] credential.driver = sql {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.902399 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.902519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.902646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.902745 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.907671 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.907797 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.907873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.907944 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.908014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.908100 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.908179 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] credential.caching = True {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.908251 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.908322 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] credential.driver = sql {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.908427 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.908505 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.908660 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.908736 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.908807 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.908878 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.908949 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.909019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.909097 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.909169 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.909240 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.909310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.909380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.909449 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.909527 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.909599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.909669 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] federation.caching = True {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.909740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.909810 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.909881 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.909959 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] federation.driver = sql {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.910031 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.910108 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.910180 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.910256 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.910375 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.910494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.910569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.910640 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.910710 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] federation.caching = True {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.910779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.910849 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.910927 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.911004 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] federation.driver = sql {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.911075 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.911146 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.911216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.911216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.911356 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.911428 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] identity.caching = True {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.911498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.911567 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.911636 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.911706 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.911783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.911855 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.911925 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.911996 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.912065 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.912135 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] identity.driver = sql {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.912212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.912284 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.912354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.912423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.912493 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] identity.caching = True {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.912562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.912638 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.912710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.912781 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.912850 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.912909 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.912978 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.913104 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.913189 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.913262 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] identity.driver = sql {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.913337 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.913409 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.913478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.913560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.913632 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.913632 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.913758 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.913829 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.913907 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.913907 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.914035 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.914106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.914175 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.914244 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.914321 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.914321 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.914522 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.914598 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.914669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.914739 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.914822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.914896 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.914966 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.915036 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.915106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.915177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.915255 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.915326 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.915396 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.915466 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.915535 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.915604 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.915681 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.915753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.915828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.915906 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.915977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.916048 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.916125 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.916197 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.916268 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.916337 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.916412 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.916483 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.916561 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.916633 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.password = **** {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.916704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.916773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.916843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.916913 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.916998 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.917071 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.917141 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.917215 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.917285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.917355 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.917433 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.917512 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.917589 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.917660 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.917730 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.917799 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.917916 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.password = **** {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.918004 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.918154 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.918265 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.918463 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.918547 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.918658 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.918739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.918892 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.918969 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.919126 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.919205 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.919357 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.user = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.919443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.919599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.919684 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.919833 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.919911 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.920071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.920163 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.920301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.920378 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.920542 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.920620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.920777 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.920856 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.921005 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.921082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.921235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.921312 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.921469 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.922044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.user = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.922227 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.922347 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.922485 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.922601 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.922731 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.922847 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.922960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.923074 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.923187 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.923299 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.923422 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.923537 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.923651 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.923764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.923878 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.923991 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.924113 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.924228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.924342 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.924455 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.924569 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.924691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.924807 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.924920 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.925034 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.925147 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.925260 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.925384 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.925500 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.925613 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.925728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.925847 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.925962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.926147 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.926264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] policy.driver = sql {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.926377 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.926512 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.926626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.926741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.926866 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.926974 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.927116 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.927213 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] receipt.caching = True {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.927320 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.927433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.927555 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.927670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.927785 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.927898 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.928011 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.928124 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.928249 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.928365 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.928478 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] policy.driver = sql {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.928591 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] resource.caching = True {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.928703 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.928818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.928939 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.928939 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.929153 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] resource.driver = sql {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.929267 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] receipt.caching = True {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.929380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.929492 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.929613 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.929728 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.929843 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.929955 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.930069 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] revoke.caching = True {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.930181 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.930303 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.930475 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.930587 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.930683 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] resource.caching = True {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.930777 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] role.cache_time = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.930777 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] role.caching = True {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.935685 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.935809 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] role.driver = sql {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.935886 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] resource.driver = sql {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.935968 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] role.list_limit = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.936041 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.936112 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.936190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.936262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.936333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.936402 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.936472 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] revoke.caching = True {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.936542 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.936620 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.936620 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.936748 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.936818 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] role.cache_time = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.936887 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.936956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] role.caching = True {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.937034 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.937105 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] role.driver = sql {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.937176 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.937264 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] role.list_limit = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.937329 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.937980 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.938067 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.938141 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.938213 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.938213 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.938339 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.938488 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.938601 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.938601 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.938739 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.938810 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.938880 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.938958 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.939031 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.939101 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.939172 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.939241 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.939311 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.939389 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.939461 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.939461 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.939587 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.939658 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.939727 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.939805 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.939878 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.939948 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.940022 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.940092 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.940162 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.940239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.940312 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.940382 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.940451 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.940569 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.940645 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.940724 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.940796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.940867 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.940937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.941007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.941077 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.941155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.941155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] token.cache_time = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.941272 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.941342 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] token.caching = True {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.941411 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.941481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.941558 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.941630 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] token.provider = fernet {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.941700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.941700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.941826 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.941895 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.941973 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.942045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.942115 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.942115 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.942240 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.942309 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.942405 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.942481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.942551 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.942621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] trust.driver = sql {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.942691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] token.cache_time = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.942762 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.942844 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] token.caching = True {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.942917 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.942987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.943057 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.943127 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] token.provider = fernet {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.943197 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.943275 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.943347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.943417 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.943487 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.943487 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.943631 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.943631 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.943766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.943839 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.943909 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] database.connection = **** {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.944057 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.944137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.944307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] trust.driver = sql {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.944395 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.944540 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.944620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.944777 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.944854 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.945007 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.945092 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.945238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.945316 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.945474 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.945551 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.945707 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.945794 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.945951 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.946038 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.946185 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.946263 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.946447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.946530 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] database.connection = **** {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.946682 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.946762 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.946934 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.947012 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.947170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.947256 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.947412 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.947492 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.947640 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.947719 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.947871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.947948 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.948103 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.948182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.948254 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=82088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 10:43:19.948325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.948557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.948789 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.949025 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.949256 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.949500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.949732 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.949972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.950198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.950457 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.950691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:43:19.950858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=82089) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 10:43:19.978288 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:19.978533 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:19.978753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:19.980495 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:19.980724 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:19.980944 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:19.987087 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:19.989713 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.014961 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.015184 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.016682 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.016913 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.075391 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.075579 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.124513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.125276 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.126448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.130008 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.295412 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.295603 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.295684 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.295750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.295859 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.296012 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.296099 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.296182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.296280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.296357 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.296484 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.296703 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.319717 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.319912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.319999 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.320094 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.320172 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.320323 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.320426 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.320547 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.320630 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.320768 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.320851 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.320980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.321062 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.321194 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.335939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.336047 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.348336 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.348564 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.348718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.348822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.348949 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.349036 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.349172 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.349254 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.349391 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.349491 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.349612 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.349694 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=82089) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.349800 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.350018 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=82088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:43:20.380059 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.380267 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 10:43:20.380619 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.380822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 10:43:20.380937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.381101 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.381184 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 10:43:20.381367 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.381475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.381675 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 10:43:20.381889 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.381986 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.382157 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.382235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.382470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.382733 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.392271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.392552 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.392843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.393134 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.393229 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.393429 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.393698 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.393977 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.400167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=82089) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.400931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=82088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.403114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=82089) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.406217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.406217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.406431 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=82088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.406548 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.406629 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.406717 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.406792 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.406864 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.406936 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.407007 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.407078 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.407156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.407230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.407230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.407359 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.407430 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.407509 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.407582 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.407653 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.407723 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.407723 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.428719 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.429013 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.430213 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=82089) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.430486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=82088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.430571 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.430769 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.435180 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=82089) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.435266 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=82088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.435470 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.435551 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.435779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.435879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.442767 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=82088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.442865 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=82089) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.443052 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.443134 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.446996 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=82088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.447236 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=82089) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.447320 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.447528 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.447608 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.447816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.447910 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.448090 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.448172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.448372 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.448657 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.448933 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.449205 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.450088 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.450381 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.450687 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.466559 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.466838 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.467071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.467353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.470668 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=82088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.471441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=82089) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.473693 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.473986 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.474691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.474993 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.478241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=82088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.482213 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=82089) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.482299 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=82088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.482417 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=82089) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.484880 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.485345 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.485636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.485919 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.486205 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.486324 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.486604 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.486696 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.486893 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.487182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.487478 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.487762 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.503126 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=82088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.503415 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.503707 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.503990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.504588 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=82089) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.504882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.505156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.505424 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.510680 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.512323 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.516661 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.516773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.516856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.516931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.517003 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.517003 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.517189 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=82088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.517263 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.517518 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.517801 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.518102 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.518746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=82089) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.519059 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.519458 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.519757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.520050 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.529327 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=82088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.529628 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.529910 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.530194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.530503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.530782 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.531093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.531329 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=82089) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.531705 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.532092 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.532406 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.532694 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.532979 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.533274 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.546309 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=82088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.549371 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=82089) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.549870 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=82088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.552370 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=82089) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.552740 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.554161 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.555312 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.555666 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=82088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.555954 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.556238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.556756 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.558362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=82089) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.558687 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.558976 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.563362 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.564838 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=82088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.567514 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.567733 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.568026 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.568305 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.568577 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.569021 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=82089) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.572030 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.572337 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.572666 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.572932 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.576445 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=82088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.576736 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.577015 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.581023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=82089) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.581313 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.581599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.584044 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=82088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.584326 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.584604 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.584876 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.585146 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.585417 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.585699 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.586019 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.586258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.586555 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.586837 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.587099 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.587360 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.587635 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=82088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.588583 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=82089) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:43:20.588872 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.589157 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.589494 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.589739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.590026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.590318 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.590624 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.590908 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.591197 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.591476 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.591742 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.592007 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.592279 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=82089) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:43:20.621893 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55775268f9c0 pid: 82088 (default app) Aug 31 10:43:20.626367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4285ec-a611-4690-8233-5b0c0e9a4ada None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:20.626732 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4285ec-a611-4690-8233-5b0c0e9a4ada None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:20.627055 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4285ec-a611-4690-8233-5b0c0e9a4ada None None] PATH_INFO: `/v3` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:20.628195 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55775268f9c0 pid: 82089 (default app) Aug 31 10:43:20.631329 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1/1] 10.222.0.16 () {50 vars in 791 bytes} [Wed Aug 31 10:43:20 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:28.577030 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422276a-1897-4135-a759-56cae30d9641 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:28.578314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422276a-1897-4135-a759-56cae30d9641 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:28.578314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422276a-1897-4135-a759-56cae30d9641 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:28.582672 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1/2] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:43:28 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:43:28.587819 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91f0f5-1f04-4260-a1c9-5a0d5dd091ba None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:28.588246 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91f0f5-1f04-4260-a1c9-5a0d5dd091ba None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:28.588413 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91f0f5-1f04-4260-a1c9-5a0d5dd091ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:28.661077 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: INFO dbcounter [None req-4e91f0f5-1f04-4260-a1c9-5a0d5dd091ba None None] Registered counter for database keystone Aug 31 10:43:28.682535 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG oslo_db.sqlalchemy.engines [None req-4e91f0f5-1f04-4260-a1c9-5a0d5dd091ba None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=82088) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 10:43:28.689527 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writer thread running {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 10:43:28.870961 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG passlib.handlers.bcrypt [None req-4e91f0f5-1f04-4260-a1c9-5a0d5dd091ba None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=82088) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 10:43:28.871274 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG passlib.handlers.bcrypt [None req-4e91f0f5-1f04-4260-a1c9-5a0d5dd091ba None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=82088) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 10:43:28.929731 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-4e91f0f5-1f04-4260-a1c9-5a0d5dd091ba None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:43:28.931396 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-4e91f0f5-1f04-4260-a1c9-5a0d5dd091ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:43:29.065308 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 2/3] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:43:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 31 10:43:29.534476 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422276a-1897-4135-a759-56cae30d9641 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:29.606745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: INFO dbcounter [None req-b422276a-1897-4135-a759-56cae30d9641 None None] Registered counter for database keystone Aug 31 10:43:29.626681 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG oslo_db.sqlalchemy.engines [None req-b422276a-1897-4135-a759-56cae30d9641 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=82089) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 10:43:29.633026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writer thread running {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 10:43:29.755666 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422276a-1897-4135-a759-56cae30d9641 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:29.759904 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e967ce-3983-488a-aba8-49da35220ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:29.760526 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e967ce-3983-488a-aba8-49da35220ecc admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:29.760800 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e967ce-3983-488a-aba8-49da35220ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:29.761065 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e967ce-3983-488a-aba8-49da35220ecc admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:29.769329 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e967ce-3983-488a-aba8-49da35220ecc admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:29.820748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: WARNING keystone.common.rbac_enforcer.enforcer [None req-55e967ce-3983-488a-aba8-49da35220ecc admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 10:43:29.821022 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e967ce-3983-488a-aba8-49da35220ecc admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:29.830879 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 2/4] 10.222.0.16 () {58 vars in 1324 bytes} [Wed Aug 31 10:43:29 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 10:43:29.837051 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e91f0f5-1f04-4260-a1c9-5a0d5dd091ba None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:29.852502 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e91f0f5-1f04-4260-a1c9-5a0d5dd091ba None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:29.856921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf8539-a7ad-490a-b1a4-7843eb1f0da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:29.858072 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf8539-a7ad-490a-b1a4-7843eb1f0da5 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:29.858349 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf8539-a7ad-490a-b1a4-7843eb1f0da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:29.859568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf8539-a7ad-490a-b1a4-7843eb1f0da5 admin admin] PATH_INFO: `/v3/projects/776925ef060044f49d8dab7d8928f73c` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:29.861153 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf8539-a7ad-490a-b1a4-7843eb1f0da5 admin admin] RBAC: Authorizing `identity:get_project(project_id=776925ef060044f49d8dab7d8928f73c)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:29.915228 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b3cf8539-a7ad-490a-b1a4-7843eb1f0da5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 10:43:29.915573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf8539-a7ad-490a-b1a4-7843eb1f0da5 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:29.920254 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 3/5] 10.222.0.16 () {56 vars in 1207 bytes} [Wed Aug 31 10:43:29 2022] GET /identity/v3/projects/776925ef060044f49d8dab7d8928f73c => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:31.809850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cb7d4-3de2-4fb8-9976-6093c44a0e6f None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:31.810408 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cb7d4-3de2-4fb8-9976-6093c44a0e6f None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:31.810408 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cb7d4-3de2-4fb8-9976-6093c44a0e6f None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:31.815299 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 3/6] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:43:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:43:31.820974 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb75a00-bb40-438f-9369-15850ca235fa None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:31.821387 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb75a00-bb40-438f-9369-15850ca235fa None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:31.821567 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb75a00-bb40-438f-9369-15850ca235fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:31.842304 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-2cb75a00-bb40-438f-9369-15850ca235fa None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:43:31.844525 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2cb75a00-bb40-438f-9369-15850ca235fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:43:31.852415 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 4/7] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:43:31 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 31 10:43:32.134284 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cb7d4-3de2-4fb8-9976-6093c44a0e6f None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:32.147328 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cb7d4-3de2-4fb8-9976-6093c44a0e6f None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:32.151760 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1409dc-d5c7-4747-996e-4036506810e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:32.152364 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1409dc-d5c7-4747-996e-4036506810e1 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:32.152642 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1409dc-d5c7-4747-996e-4036506810e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:32.152913 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1409dc-d5c7-4747-996e-4036506810e1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:32.161808 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1409dc-d5c7-4747-996e-4036506810e1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:32.163621 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1409dc-d5c7-4747-996e-4036506810e1 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:32.173772 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 4/8] 10.222.0.16 () {58 vars in 1324 bytes} [Wed Aug 31 10:43:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 10:43:32.179639 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb75a00-bb40-438f-9369-15850ca235fa None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:32.190406 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb75a00-bb40-438f-9369-15850ca235fa None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:32.194968 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b7d20-4d75-4798-bd0b-48aa7c8c25d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:32.195674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b7d20-4d75-4798-bd0b-48aa7c8c25d7 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:32.195974 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b7d20-4d75-4798-bd0b-48aa7c8c25d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:32.196247 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b7d20-4d75-4798-bd0b-48aa7c8c25d7 admin admin] PATH_INFO: `/v3/users/32a2719366d1460cb5f9b2c1c8e41774` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:32.197644 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b7d20-4d75-4798-bd0b-48aa7c8c25d7 admin admin] RBAC: Authorizing `identity:get_user(user_id=32a2719366d1460cb5f9b2c1c8e41774)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:32.199521 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b7d20-4d75-4798-bd0b-48aa7c8c25d7 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:32.201106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 5/9] 10.222.0.16 () {56 vars in 1198 bytes} [Wed Aug 31 10:43:32 2022] GET /identity/v3/users/32a2719366d1460cb5f9b2c1c8e41774 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:34.192994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40556f4a-2c55-45a5-b489-741728e38c6d None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:34.193650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40556f4a-2c55-45a5-b489-741728e38c6d None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:34.194036 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40556f4a-2c55-45a5-b489-741728e38c6d None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:34.195408 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 5/10] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:43:34.200605 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0517d5c-7218-48e6-9e71-b1ef9862e5ee None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:34.200984 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0517d5c-7218-48e6-9e71-b1ef9862e5ee None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:34.201162 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0517d5c-7218-48e6-9e71-b1ef9862e5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:34.219975 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-c0517d5c-7218-48e6-9e71-b1ef9862e5ee None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:43:34.221174 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-c0517d5c-7218-48e6-9e71-b1ef9862e5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:43:34.299853 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18b27d-c0f8-4ce5-b2fa-4d1bd98f74b2 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:34.300132 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18b27d-c0f8-4ce5-b2fa-4d1bd98f74b2 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:34.300574 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18b27d-c0f8-4ce5-b2fa-4d1bd98f74b2 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:34.301687 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 6/11] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:43:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:43:34.302020 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 6/12] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:43:34.306963 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75a401-3d02-4186-bede-04b0ebd57dd9 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:34.307246 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75a401-3d02-4186-bede-04b0ebd57dd9 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:34.307518 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75a401-3d02-4186-bede-04b0ebd57dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:34.326190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-5f75a401-3d02-4186-bede-04b0ebd57dd9 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:43:34.327993 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-5f75a401-3d02-4186-bede-04b0ebd57dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:43:34.345257 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:34.398682 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 7/13] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:43:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:43:34.586230 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18b27d-c0f8-4ce5-b2fa-4d1bd98f74b2 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:34.666621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75a401-3d02-4186-bede-04b0ebd57dd9 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:34.751187 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18b27d-c0f8-4ce5-b2fa-4d1bd98f74b2 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:34.774927 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75a401-3d02-4186-bede-04b0ebd57dd9 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:34.789141 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea6f3e-3bfc-4643-8e24-be7de493df49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:34.790498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea6f3e-3bfc-4643-8e24-be7de493df49 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:34.790919 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea6f3e-3bfc-4643-8e24-be7de493df49 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:34.791198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea6f3e-3bfc-4643-8e24-be7de493df49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:34.804129 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea6f3e-3bfc-4643-8e24-be7de493df49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:43:34.813758 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e109bdd-cf3d-45c5-997e-4d1c5a6076a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:34.814678 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e109bdd-cf3d-45c5-997e-4d1c5a6076a8 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:34.814968 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e109bdd-cf3d-45c5-997e-4d1c5a6076a8 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:34.815242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e109bdd-cf3d-45c5-997e-4d1c5a6076a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:34.815506 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-8cea6f3e-3bfc-4643-8e24-be7de493df49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:43:34.815506 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:43:34.815506 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:43:34.815506 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:43:34.815506 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:43:34.815506 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:43:34.815506 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:43:34.815506 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:43:34.815506 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:43:34.815506 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:43:34.815506 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:43:34.815506 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:43:34.815506 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:43:34.815506 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:43:34.815506 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:43:34.815506 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:43:34.816339 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:43:34.816339 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:43:34.816339 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:43:34.816339 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:43:34.816339 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:43:34.816339 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:43:34.816339 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:43:34.816339 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:43:34.816339 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:43:34.816339 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:43:34.816339 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:43:34.816339 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:43:34.816339 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:43:34.816339 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:43:34.816339 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:43:34.816339 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:43:34.817139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:43:34.817139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:43:34.817139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:43:34.817139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:43:34.817139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:43:34.817139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:43:34.817139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:43:34.817139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:43:34.817139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:43:34.817139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:43:34.817139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:43:34.817139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:43:34.817139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 7/14] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:43:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:43:34.822421 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea6f3e-3bfc-4643-8e24-be7de493df49 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:34.843260 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:34.858833 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:34.895123 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e109bdd-cf3d-45c5-997e-4d1c5a6076a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:34.896623 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e109bdd-cf3d-45c5-997e-4d1c5a6076a8 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:34.933783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea6f3e-3bfc-4643-8e24-be7de493df49 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:34.970740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbba97a-d541-465f-b92d-b8dec56d9a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:34.971357 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbba97a-d541-465f-b92d-b8dec56d9a41 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:34.971630 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbba97a-d541-465f-b92d-b8dec56d9a41 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:34.971905 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbba97a-d541-465f-b92d-b8dec56d9a41 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:34.972572 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbba97a-d541-465f-b92d-b8dec56d9a41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:43:34.972867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbba97a-d541-465f-b92d-b8dec56d9a41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:34.974274 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbba97a-d541-465f-b92d-b8dec56d9a41 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:34.986739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 8/15] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:43:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:43:34.992311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbba97a-d541-465f-b92d-b8dec56d9a41 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:35.007678 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 8/16] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:43:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 10:43:35.018320 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e109bdd-cf3d-45c5-997e-4d1c5a6076a8 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:35.073599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:35.095802 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:35.101563 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbba97a-d541-465f-b92d-b8dec56d9a41 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:35.127667 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e109bdd-cf3d-45c5-997e-4d1c5a6076a8 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:35.135651 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76846e63-4dde-4b6c-80f1-b99ed14a5094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:35.136282 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76846e63-4dde-4b6c-80f1-b99ed14a5094 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:35.136555 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76846e63-4dde-4b6c-80f1-b99ed14a5094 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:35.136856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76846e63-4dde-4b6c-80f1-b99ed14a5094 None admin] PATH_INFO: `/v3/users/32a2719366d1460cb5f9b2c1c8e41774` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:35.138441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76846e63-4dde-4b6c-80f1-b99ed14a5094 None admin] RBAC: Authorizing `identity:get_user(user_id=32a2719366d1460cb5f9b2c1c8e41774)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:35.140438 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76846e63-4dde-4b6c-80f1-b99ed14a5094 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:35.142203 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 9/17] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:43:34 2022] GET /identity/v3/users/32a2719366d1460cb5f9b2c1c8e41774 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:43:35.147588 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76846e63-4dde-4b6c-80f1-b99ed14a5094 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:35.163348 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1677040-cbfd-4990-9cdf-f74c090b541d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:35.163985 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1677040-cbfd-4990-9cdf-f74c090b541d None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:35.164267 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1677040-cbfd-4990-9cdf-f74c090b541d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:35.164545 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1677040-cbfd-4990-9cdf-f74c090b541d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:35.174892 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1677040-cbfd-4990-9cdf-f74c090b541d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:35.176382 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1677040-cbfd-4990-9cdf-f74c090b541d None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:35.195741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-f1677040-cbfd-4990-9cdf-f74c090b541d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:43:35.195741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:43:35.195741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:43:35.195741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:43:35.195741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:43:35.195741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:43:35.195741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:43:35.195741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:43:35.195741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:43:35.195741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:43:35.195741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:43:35.195741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:43:35.195741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:43:35.195741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:43:35.195741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:43:35.195741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:43:35.196593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:43:35.196593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:43:35.196593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:43:35.196593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:43:35.196593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:43:35.196593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:43:35.196593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:43:35.196593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:43:35.196593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:43:35.196593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:43:35.196593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:43:35.196593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:43:35.196593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:43:35.196593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:43:35.196593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:43:35.196593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:43:35.196593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:43:35.197393 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:43:35.197393 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:43:35.197393 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:43:35.197393 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 9/18] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:43:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:43:35.203225 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1677040-cbfd-4990-9cdf-f74c090b541d None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:35.254456 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76846e63-4dde-4b6c-80f1-b99ed14a5094 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:35.289123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292e759-7e31-4ea3-baab-4d6f01a85806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:35.289755 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292e759-7e31-4ea3-baab-4d6f01a85806 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:35.290026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292e759-7e31-4ea3-baab-4d6f01a85806 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:35.290290 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292e759-7e31-4ea3-baab-4d6f01a85806 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:35.291530 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6292e759-7e31-4ea3-baab-4d6f01a85806 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:35.292992 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6292e759-7e31-4ea3-baab-4d6f01a85806 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:35.294566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 10/19] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:43:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:43:35.300335 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292e759-7e31-4ea3-baab-4d6f01a85806 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:35.312848 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1677040-cbfd-4990-9cdf-f74c090b541d None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:35.315513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:35.332369 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:35.348220 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c851f48-dcb8-4314-914e-5b495a8b463d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:35.348845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c851f48-dcb8-4314-914e-5b495a8b463d None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:35.349127 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c851f48-dcb8-4314-914e-5b495a8b463d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:35.349400 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c851f48-dcb8-4314-914e-5b495a8b463d None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:35.350046 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c851f48-dcb8-4314-914e-5b495a8b463d None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:43:35.350348 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c851f48-dcb8-4314-914e-5b495a8b463d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:35.351820 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c851f48-dcb8-4314-914e-5b495a8b463d None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:35.378481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 10/20] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:43:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:43:35.405720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292e759-7e31-4ea3-baab-4d6f01a85806 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:35.444199 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ca7b0-ae58-4c18-ba5e-6d8ea49f8b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:35.444828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ca7b0-ae58-4c18-ba5e-6d8ea49f8b85 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:35.445109 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ca7b0-ae58-4c18-ba5e-6d8ea49f8b85 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:35.445382 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ca7b0-ae58-4c18-ba5e-6d8ea49f8b85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:35.446184 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382ca7b0-ae58-4c18-ba5e-6d8ea49f8b85 None admin] RBAC: Adding query filter params (role.id=13ca33d694c645d6920930d39dc52a0d, scope.domain.id=default, user.id=32a2719366d1460cb5f9b2c1c8e41774) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:43:35.446466 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382ca7b0-ae58-4c18-ba5e-6d8ea49f8b85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:35.447969 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382ca7b0-ae58-4c18-ba5e-6d8ea49f8b85 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:35.457666 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 11/21] 10.222.0.16 () {56 vars in 1322 bytes} [Wed Aug 31 10:43:35 2022] GET /identity/v3/role_assignments?user.id=32a2719366d1460cb5f9b2c1c8e41774&scope.domain.id=default&role.id=13ca33d694c645d6920930d39dc52a0d => generated 233 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:37.635842 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62dd39-2223-4cb3-b259-eabecff634a3 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:37.636345 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62dd39-2223-4cb3-b259-eabecff634a3 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:37.636417 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62dd39-2223-4cb3-b259-eabecff634a3 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:37.638010 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 11/22] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:43:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:43:37.643815 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29cf7c-1079-4d37-9d05-7d59f0a5bc2c None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:37.644250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29cf7c-1079-4d37-9d05-7d59f0a5bc2c None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:37.644338 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29cf7c-1079-4d37-9d05-7d59f0a5bc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:37.663481 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG passlib.handlers.bcrypt [None req-9e29cf7c-1079-4d37-9d05-7d59f0a5bc2c None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=82089) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 10:43:37.663857 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG passlib.handlers.bcrypt [None req-9e29cf7c-1079-4d37-9d05-7d59f0a5bc2c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=82089) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 10:43:37.722435 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-9e29cf7c-1079-4d37-9d05-7d59f0a5bc2c None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:43:37.724052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-9e29cf7c-1079-4d37-9d05-7d59f0a5bc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:43:37.821205 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 12/23] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:43:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:43:37.850941 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95faa52e-6287-4cfb-8a7c-38fc370a64c4 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:37.851247 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95faa52e-6287-4cfb-8a7c-38fc370a64c4 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:37.851486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95faa52e-6287-4cfb-8a7c-38fc370a64c4 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:37.853042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 12/24] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:43:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:43:37.872698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aadf9f-f7ca-4878-a130-83dac2a7b068 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:37.872987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aadf9f-f7ca-4878-a130-83dac2a7b068 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:37.873251 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aadf9f-f7ca-4878-a130-83dac2a7b068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:37.906568 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-26aadf9f-f7ca-4878-a130-83dac2a7b068 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:43:37.907686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-26aadf9f-f7ca-4878-a130-83dac2a7b068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:43:37.929492 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:37.984037 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5a479-914f-49e5-a0e6-5eee6fd880d7 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:37.984350 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5a479-914f-49e5-a0e6-5eee6fd880d7 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:37.984579 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5a479-914f-49e5-a0e6-5eee6fd880d7 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:37.986158 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 13/25] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:43:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:43:37.992816 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2eac28-a88d-43e5-b73e-8e118b81f37a None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:37.993091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2eac28-a88d-43e5-b73e-8e118b81f37a None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:37.993360 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2eac28-a88d-43e5-b73e-8e118b81f37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:38.008120 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 13/26] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:43:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:43:38.020590 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-fe2eac28-a88d-43e5-b73e-8e118b81f37a None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:43:38.021876 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-fe2eac28-a88d-43e5-b73e-8e118b81f37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:43:38.125533 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 14/27] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:43:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:43:38.383768 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aadf9f-f7ca-4878-a130-83dac2a7b068 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:38.395638 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b07e5-cbad-4ce1-845c-d7dfb094ccc0 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:38.397867 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b07e5-cbad-4ce1-845c-d7dfb094ccc0 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:38.398369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b07e5-cbad-4ce1-845c-d7dfb094ccc0 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:38.406048 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 15/28] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:43:38 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:43:38.412020 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a36ae3-ed38-4733-8106-9607588a88e4 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:38.412532 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a36ae3-ed38-4733-8106-9607588a88e4 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:38.413015 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a36ae3-ed38-4733-8106-9607588a88e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:38.448158 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-f0a36ae3-ed38-4733-8106-9607588a88e4 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:43:38.451690 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-f0a36ae3-ed38-4733-8106-9607588a88e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:43:38.567802 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aadf9f-f7ca-4878-a130-83dac2a7b068 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:38.574049 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 16/29] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:43:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:43:38.575162 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a36ae3-ed38-4733-8106-9607588a88e4 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:38.625845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:38.632231 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68ba1a-ba81-476e-912d-82ebd890a133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:38.633225 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68ba1a-ba81-476e-912d-82ebd890a133 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:38.643912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68ba1a-ba81-476e-912d-82ebd890a133 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:38.643912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68ba1a-ba81-476e-912d-82ebd890a133 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:38.657704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68ba1a-ba81-476e-912d-82ebd890a133 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:43:38.664823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-0f68ba1a-ba81-476e-912d-82ebd890a133 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:43:38.664823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:43:38.664823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:43:38.664823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:43:38.664823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:43:38.664823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:43:38.664823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:43:38.664823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:43:38.664823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:43:38.664823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:43:38.664823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:43:38.664823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:43:38.664823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:43:38.664823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:43:38.664823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:43:38.664823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:43:38.665664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:43:38.665664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:43:38.665664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:43:38.665664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:43:38.665664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:43:38.665664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:43:38.665664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:43:38.665664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:43:38.665664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:43:38.665664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:43:38.665664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:43:38.665664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:43:38.665664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:43:38.665664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:43:38.665664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:43:38.665664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:43:38.666405 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:43:38.666405 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:43:38.666405 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:43:38.666405 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:43:38.666405 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:43:38.666405 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:43:38.666405 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:43:38.666405 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:43:38.666405 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:43:38.666405 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:43:38.666405 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:43:38.666405 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:43:38.669926 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 14/30] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:43:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:43:38.672429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68ba1a-ba81-476e-912d-82ebd890a133 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:38.708528 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:38.785014 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a36ae3-ed38-4733-8106-9607588a88e4 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:38.840730 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68ba1a-ba81-476e-912d-82ebd890a133 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:38.878248 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f463ab-3022-475d-949d-dd2649e5fbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:38.879258 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f463ab-3022-475d-949d-dd2649e5fbd7 None admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:38.879739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f463ab-3022-475d-949d-dd2649e5fbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:38.880214 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f463ab-3022-475d-949d-dd2649e5fbd7 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:38.881065 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f463ab-3022-475d-949d-dd2649e5fbd7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:38.882721 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f463ab-3022-475d-949d-dd2649e5fbd7 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:38.887310 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf647cf9-2f65-4b4a-8188-2b0dfaabcc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:38.887981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf647cf9-2f65-4b4a-8188-2b0dfaabcc78 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:38.888265 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf647cf9-2f65-4b4a-8188-2b0dfaabcc78 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:38.888539 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf647cf9-2f65-4b4a-8188-2b0dfaabcc78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:38.911470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf647cf9-2f65-4b4a-8188-2b0dfaabcc78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:38.912872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf647cf9-2f65-4b4a-8188-2b0dfaabcc78 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:38.930056 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-cf647cf9-2f65-4b4a-8188-2b0dfaabcc78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:43:38.930056 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:43:38.930056 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:43:38.930056 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:43:38.930056 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:43:38.930056 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:43:38.930056 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:43:38.930056 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:43:38.930056 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:43:38.930056 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:43:38.930056 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:43:38.930056 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:43:38.930056 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:43:38.930056 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:43:38.930056 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:43:38.930056 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:43:38.931038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:43:38.931038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:43:38.931038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:43:38.931038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:43:38.931038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:43:38.931038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:43:38.931038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:43:38.931038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:43:38.931038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:43:38.931038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:43:38.931038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:43:38.931038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:43:38.931038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:43:38.931038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:43:38.931038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:43:38.931038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:43:38.931038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:43:38.931835 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:43:38.931835 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:43:38.931835 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:43:38.931835 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 17/31] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:43:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:43:38.932067 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf647cf9-2f65-4b4a-8188-2b0dfaabcc78 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:39.029573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:39.056258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf647cf9-2f65-4b4a-8188-2b0dfaabcc78 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:39.090038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5e1be-7f46-478d-87a5-98fcfb05ed3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:39.090674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5e1be-7f46-478d-87a5-98fcfb05ed3f None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:39.090955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5e1be-7f46-478d-87a5-98fcfb05ed3f None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:39.091228 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5e1be-7f46-478d-87a5-98fcfb05ed3f None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:39.094584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5e1be-7f46-478d-87a5-98fcfb05ed3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:43:39.094584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5e1be-7f46-478d-87a5-98fcfb05ed3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:39.094584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5e1be-7f46-478d-87a5-98fcfb05ed3f None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:39.111267 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 18/32] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:43:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:39.112283 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 15/33] 10.222.0.16 () {60 vars in 1131 bytes} [Wed Aug 31 10:43:38 2022] POST /identity/v3/roles => generated 219 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:43:39.122624 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f463ab-3022-475d-949d-dd2649e5fbd7 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:39.122707 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5e1be-7f46-478d-87a5-98fcfb05ed3f None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:39.216400 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:39.247006 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f463ab-3022-475d-949d-dd2649e5fbd7 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:39.276706 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5e1be-7f46-478d-87a5-98fcfb05ed3f None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:39.291765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454d482-c374-4cc8-86d3-91a226b2f0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:39.292471 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454d482-c374-4cc8-86d3-91a226b2f0fb None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:39.292752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454d482-c374-4cc8-86d3-91a226b2f0fb None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:39.293027 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454d482-c374-4cc8-86d3-91a226b2f0fb None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:39.293680 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8454d482-c374-4cc8-86d3-91a226b2f0fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:43:39.293983 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8454d482-c374-4cc8-86d3-91a226b2f0fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:39.295541 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8454d482-c374-4cc8-86d3-91a226b2f0fb None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:39.317370 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c75b7a1-1388-49cc-bcca-08fe92adb4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:39.318434 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 16/34] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:39.318595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75b7a1-1388-49cc-bcca-08fe92adb4b1 None admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:39.318595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75b7a1-1388-49cc-bcca-08fe92adb4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:39.318595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75b7a1-1388-49cc-bcca-08fe92adb4b1 None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:39.319671 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c75b7a1-1388-49cc-bcca-08fe92adb4b1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:39.320702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c75b7a1-1388-49cc-bcca-08fe92adb4b1 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:39.324046 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac13af1-a4b6-4180-a11a-316d6339aa57 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:39.324344 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac13af1-a4b6-4180-a11a-316d6339aa57 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:39.324625 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac13af1-a4b6-4180-a11a-316d6339aa57 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:39.326277 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 17/35] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:43:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:43:39.327492 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac13af1-a4b6-4180-a11a-316d6339aa57 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:39.408093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:39.431558 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac13af1-a4b6-4180-a11a-316d6339aa57 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:39.466347 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c1065d-5538-4e5b-b3e2-38de7e189e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:39.467080 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1065d-5538-4e5b-b3e2-38de7e189e38 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:39.467313 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1065d-5538-4e5b-b3e2-38de7e189e38 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:39.468194 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1065d-5538-4e5b-b3e2-38de7e189e38 None admin] PATH_INFO: `/v3/users/32a2719366d1460cb5f9b2c1c8e41774` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:39.469807 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c1065d-5538-4e5b-b3e2-38de7e189e38 None admin] RBAC: Authorizing `identity:get_user(user_id=32a2719366d1460cb5f9b2c1c8e41774)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:39.471293 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c1065d-5538-4e5b-b3e2-38de7e189e38 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:39.473026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 18/36] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:43:39 2022] GET /identity/v3/users/32a2719366d1460cb5f9b2c1c8e41774 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:39.474127 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f1673-2dc6-4833-83c6-9f4a07a8b78a None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:39.474422 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f1673-2dc6-4833-83c6-9f4a07a8b78a None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:39.474703 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f1673-2dc6-4833-83c6-9f4a07a8b78a None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:39.476059 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 19/37] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:43:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:43:39.478841 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f1673-2dc6-4833-83c6-9f4a07a8b78a None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:39.482431 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:39.523569 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 19/38] 10.222.0.16 () {60 vars in 1131 bytes} [Wed Aug 31 10:43:39 2022] POST /identity/v3/roles => generated 225 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:43:39.525624 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96739d07-ce0c-49ef-8b44-b2b4f0f4b94a None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:39.525919 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96739d07-ce0c-49ef-8b44-b2b4f0f4b94a None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:39.526197 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96739d07-ce0c-49ef-8b44-b2b4f0f4b94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:39.547347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-96739d07-ce0c-49ef-8b44-b2b4f0f4b94a None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:43:39.548947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-96739d07-ce0c-49ef-8b44-b2b4f0f4b94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:43:39.597210 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f1673-2dc6-4833-83c6-9f4a07a8b78a None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:39.629206 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 20/39] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:43:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:43:39.629895 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96739d07-ce0c-49ef-8b44-b2b4f0f4b94a None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:39.659229 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c963b65a-4c44-409b-9361-27a100173b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:39.659947 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963b65a-4c44-409b-9361-27a100173b23 None admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:39.660192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963b65a-4c44-409b-9361-27a100173b23 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:39.660470 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963b65a-4c44-409b-9361-27a100173b23 None admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:39.661154 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c963b65a-4c44-409b-9361-27a100173b23 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:39.662718 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c963b65a-4c44-409b-9361-27a100173b23 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:39.751981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96739d07-ce0c-49ef-8b44-b2b4f0f4b94a None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:39.784660 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 20/40] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Aug 31 10:43:39 2022] POST /identity/v3/projects => generated 301 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:43:39.785980 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4ce15-578d-40b0-8ccf-2c0356195270 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:39.786278 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4ce15-578d-40b0-8ccf-2c0356195270 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:39.786596 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4ce15-578d-40b0-8ccf-2c0356195270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:39.813022 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-24b4ce15-578d-40b0-8ccf-2c0356195270 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:43:39.814974 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-24b4ce15-578d-40b0-8ccf-2c0356195270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:43:39.885882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:39.893567 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:39.913054 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac311c1-9175-4eb2-b85c-3bfaa964894d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:39.913054 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac311c1-9175-4eb2-b85c-3bfaa964894d None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:39.913054 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac311c1-9175-4eb2-b85c-3bfaa964894d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:39.913054 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac311c1-9175-4eb2-b85c-3bfaa964894d None admin] PATH_INFO: `/v3/domains/default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:39.913450 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac311c1-9175-4eb2-b85c-3bfaa964894d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:39.913450 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac311c1-9175-4eb2-b85c-3bfaa964894d None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:39.913450 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 21/41] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:43:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:39.913674 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 21/42] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:43:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:43:39.915041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac311c1-9175-4eb2-b85c-3bfaa964894d None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:39.959122 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4ce15-578d-40b0-8ccf-2c0356195270 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:40.058559 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac311c1-9175-4eb2-b85c-3bfaa964894d None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:40.083213 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4ce15-578d-40b0-8ccf-2c0356195270 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:40.095386 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b89c87-59b0-4bcb-9931-072b92197260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:40.096291 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b89c87-59b0-4bcb-9931-072b92197260 None admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:40.096718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b89c87-59b0-4bcb-9931-072b92197260 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:40.097203 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b89c87-59b0-4bcb-9931-072b92197260 None admin] PATH_INFO: `/v3/domains/default/users/32a2719366d1460cb5f9b2c1c8e41774/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:40.099531 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b89c87-59b0-4bcb-9931-072b92197260 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=32a2719366d1460cb5f9b2c1c8e41774, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:40.101013 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b89c87-59b0-4bcb-9931-072b92197260 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:40.124411 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0623d5-d463-4435-9ca0-0e06b6d989bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:40.125105 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0623d5-d463-4435-9ca0-0e06b6d989bc None admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:40.125378 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0623d5-d463-4435-9ca0-0e06b6d989bc None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:40.125647 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0623d5-d463-4435-9ca0-0e06b6d989bc None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:40.126323 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0623d5-d463-4435-9ca0-0e06b6d989bc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:40.127741 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0623d5-d463-4435-9ca0-0e06b6d989bc None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:40.193684 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 22/43] 10.222.0.16 () {58 vars in 1361 bytes} [Wed Aug 31 10:43:39 2022] PUT /identity/v3/domains/default/users/32a2719366d1460cb5f9b2c1c8e41774/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:43:40.194823 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b89c87-59b0-4bcb-9931-072b92197260 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:40.238199 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 22/44] 10.222.0.16 () {60 vars in 1131 bytes} [Wed Aug 31 10:43:39 2022] POST /identity/v3/roles => generated 223 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 10:43:40.263360 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:40.316745 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b89c87-59b0-4bcb-9931-072b92197260 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:40.358908 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d9b6e-bf2d-4f0a-8f46-30a9b2d90bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:40.360054 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d9b6e-bf2d-4f0a-8f46-30a9b2d90bcb None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:40.360577 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d9b6e-bf2d-4f0a-8f46-30a9b2d90bcb None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:40.360851 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d9b6e-bf2d-4f0a-8f46-30a9b2d90bcb None admin] PATH_INFO: `/v3/domains/default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:40.362111 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03d9b6e-bf2d-4f0a-8f46-30a9b2d90bcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:40.363764 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03d9b6e-bf2d-4f0a-8f46-30a9b2d90bcb None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:40.365747 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 23/45] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:43:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:40.371316 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0623d5-d463-4435-9ca0-0e06b6d989bc None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:40.429267 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:40.484213 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0623d5-d463-4435-9ca0-0e06b6d989bc None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:40.519570 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ffd176-243e-4c1f-bf35-738b6e43bd6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:40.520192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffd176-243e-4c1f-bf35-738b6e43bd6e None admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:40.520475 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffd176-243e-4c1f-bf35-738b6e43bd6e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:40.520752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffd176-243e-4c1f-bf35-738b6e43bd6e None admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:40.521418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ffd176-243e-4c1f-bf35-738b6e43bd6e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:40.522930 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ffd176-243e-4c1f-bf35-738b6e43bd6e None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:40.606203 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 23/46] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Aug 31 10:43:40 2022] POST /identity/v3/projects => generated 312 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:43:42.140119 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c728ee6-72db-4898-be65-834e7ee299f6 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:42.140655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c728ee6-72db-4898-be65-834e7ee299f6 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:42.140655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c728ee6-72db-4898-be65-834e7ee299f6 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:42.142053 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 24/47] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:43:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:43:42.147206 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe51da-509f-4a6f-bd56-8c5f9e1e385d None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:42.147609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe51da-509f-4a6f-bd56-8c5f9e1e385d None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:42.147768 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe51da-509f-4a6f-bd56-8c5f9e1e385d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:42.165914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-8dfe51da-509f-4a6f-bd56-8c5f9e1e385d None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:43:42.167134 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-8dfe51da-509f-4a6f-bd56-8c5f9e1e385d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:43:42.244569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 24/48] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:43:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:43:42.516400 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3524f2-568a-434b-bab2-5cf34702c56f None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:42.516685 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3524f2-568a-434b-bab2-5cf34702c56f None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:42.516956 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3524f2-568a-434b-bab2-5cf34702c56f None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:42.518330 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 25/49] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:43:42.518925 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe51da-509f-4a6f-bd56-8c5f9e1e385d None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:42.523391 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0582d60-c44b-49ac-8bf4-b429bf7dbb46 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:42.523672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0582d60-c44b-49ac-8bf4-b429bf7dbb46 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:42.523947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0582d60-c44b-49ac-8bf4-b429bf7dbb46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:42.541521 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-b0582d60-c44b-49ac-8bf4-b429bf7dbb46 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:43:42.542743 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-b0582d60-c44b-49ac-8bf4-b429bf7dbb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:43:42.548067 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:42.612028 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 26/50] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:43:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:43:42.623016 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe51da-509f-4a6f-bd56-8c5f9e1e385d None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:42.659313 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d50ff-e327-4e60-9dc4-4926f9816331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:42.659949 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d50ff-e327-4e60-9dc4-4926f9816331 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:42.660228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d50ff-e327-4e60-9dc4-4926f9816331 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:42.660502 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d50ff-e327-4e60-9dc4-4926f9816331 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:42.671311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d50ff-e327-4e60-9dc4-4926f9816331 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:43:42.676207 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-3a3d50ff-e327-4e60-9dc4-4926f9816331 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:43:42.676207 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:43:42.676207 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:43:42.676207 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:43:42.676207 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:43:42.676207 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:43:42.676207 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:43:42.676207 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:43:42.676207 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:43:42.676207 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:43:42.676207 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:43:42.676207 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:43:42.676207 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:43:42.676207 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:43:42.676207 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:43:42.676207 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:43:42.677135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:43:42.677135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:43:42.677135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:43:42.677135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:43:42.677135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:43:42.677135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:43:42.677135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:43:42.677135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:43:42.677135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:43:42.677135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:43:42.677135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:43:42.677135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:43:42.677135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:43:42.677135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:43:42.677135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:43:42.677135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:43:42.677880 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:43:42.677880 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:43:42.677880 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:43:42.677880 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:43:42.677880 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:43:42.677880 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:43:42.677880 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:43:42.677880 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:43:42.677880 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:43:42.677880 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:43:42.677880 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:43:42.677880 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:43:42.677880 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 25/51] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:43:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:43:42.685932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0582d60-c44b-49ac-8bf4-b429bf7dbb46 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:42.742707 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:42.787420 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0582d60-c44b-49ac-8bf4-b429bf7dbb46 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:42.820709 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3890383-dd5f-4dd7-8dab-52d54934b56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:42.821326 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3890383-dd5f-4dd7-8dab-52d54934b56f None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:42.821606 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3890383-dd5f-4dd7-8dab-52d54934b56f None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:42.821880 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3890383-dd5f-4dd7-8dab-52d54934b56f None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:42.822590 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3890383-dd5f-4dd7-8dab-52d54934b56f None admin] RBAC: Adding query filter params (name=admin) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:43:42.822897 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3890383-dd5f-4dd7-8dab-52d54934b56f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:42.824291 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3890383-dd5f-4dd7-8dab-52d54934b56f None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:42.832390 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 27/52] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:43:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:42.838056 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d50ff-e327-4e60-9dc4-4926f9816331 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:42.884939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3890383-dd5f-4dd7-8dab-52d54934b56f None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:42.994119 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3890383-dd5f-4dd7-8dab-52d54934b56f None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:43.010587 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:43.030508 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27df28-6bab-4990-822e-79df8f275c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:43.031160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27df28-6bab-4990-822e-79df8f275c9d None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:43.031444 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27df28-6bab-4990-822e-79df8f275c9d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:43.032650 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27df28-6bab-4990-822e-79df8f275c9d None admin] PATH_INFO: `/v3/domains/default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:43.034475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27df28-6bab-4990-822e-79df8f275c9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:43.036710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27df28-6bab-4990-822e-79df8f275c9d None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:43.037336 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:43.039078 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 28/53] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:43:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:43:43.044417 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27df28-6bab-4990-822e-79df8f275c9d None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:43.051154 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d50ff-e327-4e60-9dc4-4926f9816331 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:43.085978 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce4b396-d014-45a8-9736-16fc5ed8cff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:43.086754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4b396-d014-45a8-9736-16fc5ed8cff6 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:43.087037 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4b396-d014-45a8-9736-16fc5ed8cff6 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:43.087373 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4b396-d014-45a8-9736-16fc5ed8cff6 None admin] PATH_INFO: `/v3/users/32a2719366d1460cb5f9b2c1c8e41774` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:43.088586 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce4b396-d014-45a8-9736-16fc5ed8cff6 None admin] RBAC: Authorizing `identity:get_user(user_id=32a2719366d1460cb5f9b2c1c8e41774)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:43.090159 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce4b396-d014-45a8-9736-16fc5ed8cff6 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:43.091910 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 26/54] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:43:42 2022] GET /identity/v3/users/32a2719366d1460cb5f9b2c1c8e41774 => generated 247 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:43.098252 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce4b396-d014-45a8-9736-16fc5ed8cff6 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:43.146907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27df28-6bab-4990-822e-79df8f275c9d None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:43.181186 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a3dab8-3d9e-4564-afef-32dccf24250c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:43.181790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3dab8-3d9e-4564-afef-32dccf24250c None admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:43.182069 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3dab8-3d9e-4564-afef-32dccf24250c None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:43.182342 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3dab8-3d9e-4564-afef-32dccf24250c None admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:43.183015 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a3dab8-3d9e-4564-afef-32dccf24250c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:43.184453 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a3dab8-3d9e-4564-afef-32dccf24250c None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:43.199314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce4b396-d014-45a8-9736-16fc5ed8cff6 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:43.234486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b7776-fffc-4f55-9060-b9bbc72a4557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:43.235106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b7776-fffc-4f55-9060-b9bbc72a4557 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:43.235382 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b7776-fffc-4f55-9060-b9bbc72a4557 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:43.235651 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b7776-fffc-4f55-9060-b9bbc72a4557 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:43.236760 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b7776-fffc-4f55-9060-b9bbc72a4557 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:43.238191 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b7776-fffc-4f55-9060-b9bbc72a4557 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:43.239772 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 27/55] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:43:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:43.245512 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b7776-fffc-4f55-9060-b9bbc72a4557 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:43.270341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:43.286358 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 29/56] 10.222.0.16 () {60 vars in 1140 bytes} [Wed Aug 31 10:43:43 2022] POST /identity/v3/projects => generated 298 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 10:43:43.351236 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b7776-fffc-4f55-9060-b9bbc72a4557 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:43.387855 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30930bce-6f3a-4f25-94c5-c9ebbf9d041a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:43.388486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30930bce-6f3a-4f25-94c5-c9ebbf9d041a None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:43.388768 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30930bce-6f3a-4f25-94c5-c9ebbf9d041a None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:43.389045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30930bce-6f3a-4f25-94c5-c9ebbf9d041a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:43.390541 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30930bce-6f3a-4f25-94c5-c9ebbf9d041a None admin] RBAC: Adding query filter params (role.id=13ca33d694c645d6920930d39dc52a0d, scope.domain.id=default, user.id=32a2719366d1460cb5f9b2c1c8e41774) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:43:43.390811 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30930bce-6f3a-4f25-94c5-c9ebbf9d041a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:43.392295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30930bce-6f3a-4f25-94c5-c9ebbf9d041a None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:43.398976 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 28/57] 10.222.0.16 () {56 vars in 1322 bytes} [Wed Aug 31 10:43:43 2022] GET /identity/v3/role_assignments?user.id=32a2719366d1460cb5f9b2c1c8e41774&scope.domain.id=default&role.id=13ca33d694c645d6920930d39dc52a0d => generated 530 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:43:45.188993 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72031134-8110-4270-9ab1-ce939405e51d None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:45.189500 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72031134-8110-4270-9ab1-ce939405e51d None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:45.189607 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72031134-8110-4270-9ab1-ce939405e51d None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:45.190925 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 30/58] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:43:45.195943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c751e-380e-4f73-ae4e-def5b0b5bf7f None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:45.196355 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c751e-380e-4f73-ae4e-def5b0b5bf7f None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:45.196490 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c751e-380e-4f73-ae4e-def5b0b5bf7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:45.214780 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-ea9c751e-380e-4f73-ae4e-def5b0b5bf7f None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:43:45.215949 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-ea9c751e-380e-4f73-ae4e-def5b0b5bf7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:43:45.286071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 29/59] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:43:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:43:45.557380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72031134-8110-4270-9ab1-ce939405e51d None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:45.593434 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:45.660661 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72031134-8110-4270-9ab1-ce939405e51d None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:45.693785 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d1e08-dd27-4fb9-ba52-fc6474da5d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:45.694422 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d1e08-dd27-4fb9-ba52-fc6474da5d6e None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:45.694702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d1e08-dd27-4fb9-ba52-fc6474da5d6e None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:45.694974 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d1e08-dd27-4fb9-ba52-fc6474da5d6e None admin] PATH_INFO: `/v3/domains/default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:45.696080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d1e08-dd27-4fb9-ba52-fc6474da5d6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:45.697536 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d1e08-dd27-4fb9-ba52-fc6474da5d6e None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:45.699106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 31/60] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:43:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:45.704659 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c751e-380e-4f73-ae4e-def5b0b5bf7f None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:45.723253 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:45.803572 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c751e-380e-4f73-ae4e-def5b0b5bf7f None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:45.836771 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974c397-ff5b-44a1-b378-8e77aabd05a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:45.837301 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974c397-ff5b-44a1-b378-8e77aabd05a2 None admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:45.837574 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974c397-ff5b-44a1-b378-8e77aabd05a2 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:45.837839 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974c397-ff5b-44a1-b378-8e77aabd05a2 None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:45.838523 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974c397-ff5b-44a1-b378-8e77aabd05a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:45.839961 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974c397-ff5b-44a1-b378-8e77aabd05a2 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:45.940736 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 30/61] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 31 10:43:45 2022] POST /identity/v3/users => generated 275 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 10:43:48.975221 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c4722-227c-494f-89ec-f96a7e9f56a3 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:48.975753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c4722-227c-494f-89ec-f96a7e9f56a3 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:48.975753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c4722-227c-494f-89ec-f96a7e9f56a3 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:48.977077 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 32/62] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:43:48.982347 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d37030-968e-4afe-ae1e-bd1d1dbc2d66 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:48.983207 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d37030-968e-4afe-ae1e-bd1d1dbc2d66 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:48.983702 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d37030-968e-4afe-ae1e-bd1d1dbc2d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:49.016520 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-a9d37030-968e-4afe-ae1e-bd1d1dbc2d66 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:43:49.018060 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a9d37030-968e-4afe-ae1e-bd1d1dbc2d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:43:49.143428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 31/63] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:43:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:43:49.333043 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb5731-8f32-46f5-8d9a-c8964d2c89ef None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:49.333683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb5731-8f32-46f5-8d9a-c8964d2c89ef None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:49.334162 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb5731-8f32-46f5-8d9a-c8964d2c89ef None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:49.335828 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 33/64] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:43:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:43:49.341364 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58247c7-b04c-48dc-b5c7-e1f40f278f75 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:49.341922 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58247c7-b04c-48dc-b5c7-e1f40f278f75 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:49.342415 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58247c7-b04c-48dc-b5c7-e1f40f278f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:49.353136 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:49.365089 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-d58247c7-b04c-48dc-b5c7-e1f40f278f75 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:43:49.367796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-d58247c7-b04c-48dc-b5c7-e1f40f278f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:43:49.384331 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8f537-1497-4271-898c-2a005a7fb86b None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:49.384635 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8f537-1497-4271-898c-2a005a7fb86b None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:49.384885 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8f537-1497-4271-898c-2a005a7fb86b None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:49.386460 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 34/65] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:43:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:43:49.392976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4306af7-ebe7-4cc9-a4a3-d49f13cd90e4 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:49.393262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4306af7-ebe7-4cc9-a4a3-d49f13cd90e4 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:49.393539 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4306af7-ebe7-4cc9-a4a3-d49f13cd90e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:49.419277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-f4306af7-ebe7-4cc9-a4a3-d49f13cd90e4 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:43:49.420640 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-f4306af7-ebe7-4cc9-a4a3-d49f13cd90e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:43:49.484642 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 32/66] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:43:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:43:49.548863 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 35/67] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:43:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:43:49.572498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb879d-89ad-43bc-8467-a57d4bf30c36 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:49.572792 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb879d-89ad-43bc-8467-a57d4bf30c36 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:49.573060 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb879d-89ad-43bc-8467-a57d4bf30c36 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:49.574569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 33/68] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:43:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:43:49.580582 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e763feb-08fd-4909-9cc6-8ea1e3a5f6f7 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:49.581130 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e763feb-08fd-4909-9cc6-8ea1e3a5f6f7 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:49.581600 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e763feb-08fd-4909-9cc6-8ea1e3a5f6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:49.601290 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-9e763feb-08fd-4909-9cc6-8ea1e3a5f6f7 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:43:49.602826 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-9e763feb-08fd-4909-9cc6-8ea1e3a5f6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:43:49.629469 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb879d-89ad-43bc-8467-a57d4bf30c36 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:49.635125 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:49.710090 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 36/69] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:43:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:43:49.716661 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001de08c-5c47-43c6-8034-eb90530feec4 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:49.716661 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001de08c-5c47-43c6-8034-eb90530feec4 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:49.716661 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001de08c-5c47-43c6-8034-eb90530feec4 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:49.716661 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 37/70] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:43:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:43:49.732371 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb61a4-b916-4ee8-ae45-848721caa7a4 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:49.732658 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb61a4-b916-4ee8-ae45-848721caa7a4 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:49.732938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb61a4-b916-4ee8-ae45-848721caa7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:49.763643 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-e8eb61a4-b916-4ee8-ae45-848721caa7a4 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:43:49.764874 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-e8eb61a4-b916-4ee8-ae45-848721caa7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:43:49.796947 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb879d-89ad-43bc-8467-a57d4bf30c36 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:49.835003 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435086af-7ce3-490d-b9af-31e40287f2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:49.836141 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435086af-7ce3-490d-b9af-31e40287f2f1 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:49.836674 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435086af-7ce3-490d-b9af-31e40287f2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:49.837251 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435086af-7ce3-490d-b9af-31e40287f2f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:49.839623 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:49.851002 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435086af-7ce3-490d-b9af-31e40287f2f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:43:49.857440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-435086af-7ce3-490d-b9af-31e40287f2f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:43:49.857440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:43:49.857440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:43:49.857440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:43:49.857440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:43:49.857440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:43:49.857440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:43:49.857440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:43:49.857440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:43:49.857440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:43:49.857440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:43:49.857440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:43:49.857440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:43:49.857440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:43:49.857440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:43:49.857440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:43:49.858271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:43:49.858271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:43:49.858271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:43:49.858271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:43:49.858271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:43:49.858271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:43:49.858271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:43:49.858271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:43:49.858271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:43:49.858271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:43:49.858271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:43:49.858271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:43:49.858271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:43:49.858271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:43:49.858271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:43:49.858271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:43:49.859154 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:43:49.859154 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:43:49.859154 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:43:49.859154 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:43:49.859154 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:43:49.859154 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:43:49.859154 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:43:49.859154 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:43:49.859154 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:43:49.859154 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:43:49.859154 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:43:49.859154 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:43:49.861805 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 34/71] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:43:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:43:49.862920 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435086af-7ce3-490d-b9af-31e40287f2f1 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:49.867362 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 38/72] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:43:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:43:49.874792 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb61a4-b916-4ee8-ae45-848721caa7a4 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:49.977289 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:49.977894 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435086af-7ce3-490d-b9af-31e40287f2f1 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:49.985802 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb61a4-b916-4ee8-ae45-848721caa7a4 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:50.013037 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1e756-7af6-4af4-925a-2ba41aa5cb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:50.014314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1e756-7af6-4af4-925a-2ba41aa5cb3e None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:50.014779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1e756-7af6-4af4-925a-2ba41aa5cb3e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:50.015864 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1e756-7af6-4af4-925a-2ba41aa5cb3e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:50.025443 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e611286-0afd-48d4-b3a6-9426bf62f5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:50.026614 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e611286-0afd-48d4-b3a6-9426bf62f5af None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:50.027164 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e611286-0afd-48d4-b3a6-9426bf62f5af None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:50.027266 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b1e756-7af6-4af4-925a-2ba41aa5cb3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:43:50.027704 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e611286-0afd-48d4-b3a6-9426bf62f5af None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:50.028663 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e611286-0afd-48d4-b3a6-9426bf62f5af None admin] RBAC: Adding query filter params (name=member) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:43:50.028974 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e611286-0afd-48d4-b3a6-9426bf62f5af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:50.030414 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e611286-0afd-48d4-b3a6-9426bf62f5af None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:50.032956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-62b1e756-7af6-4af4-925a-2ba41aa5cb3e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:43:50.032956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:43:50.032956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:43:50.032956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:43:50.032956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:43:50.032956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:43:50.032956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:43:50.032956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:43:50.032956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:43:50.032956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:43:50.032956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:43:50.032956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:43:50.032956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:43:50.032956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:43:50.032956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:43:50.032956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:43:50.033756 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:43:50.033756 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:43:50.033756 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:43:50.033756 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:43:50.033756 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:43:50.033756 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:43:50.033756 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:43:50.033756 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:43:50.033756 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:43:50.033756 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:43:50.033756 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:43:50.033756 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:43:50.033756 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:43:50.033756 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:43:50.033756 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:43:50.033756 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:43:50.034937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:43:50.034937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:43:50.034937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:43:50.034937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:43:50.034937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:43:50.034937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:43:50.034937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:43:50.034937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:43:50.034937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:43:50.034937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:43:50.034937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:43:50.034937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:43:50.034937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 35/73] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 31 10:43:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:43:50.034937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1e756-7af6-4af4-925a-2ba41aa5cb3e None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:50.040782 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 39/74] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 31 10:43:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:50.041538 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e611286-0afd-48d4-b3a6-9426bf62f5af None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:50.109944 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:50.156496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1e756-7af6-4af4-925a-2ba41aa5cb3e None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:50.156830 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e611286-0afd-48d4-b3a6-9426bf62f5af None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:50.190987 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5cb68e-4cec-42ea-bbb8-362220cb8677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:50.191619 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5cb68e-4cec-42ea-bbb8-362220cb8677 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:50.191744 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd6ee5-77c3-4202-af6f-2ff3d08c77fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:50.191899 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5cb68e-4cec-42ea-bbb8-362220cb8677 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:50.192171 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5cb68e-4cec-42ea-bbb8-362220cb8677 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:50.192369 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd6ee5-77c3-4202-af6f-2ff3d08c77fd None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:50.192646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd6ee5-77c3-4202-af6f-2ff3d08c77fd None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:50.192914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd6ee5-77c3-4202-af6f-2ff3d08c77fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:50.203186 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd6ee5-77c3-4202-af6f-2ff3d08c77fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:43:50.204531 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5cb68e-4cec-42ea-bbb8-362220cb8677 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:43:50.207949 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-c6fd6ee5-77c3-4202-af6f-2ff3d08c77fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:43:50.207949 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:43:50.207949 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:43:50.207949 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:43:50.207949 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:43:50.207949 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:43:50.207949 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:43:50.207949 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:43:50.207949 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:43:50.207949 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:43:50.207949 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:43:50.207949 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:43:50.207949 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:43:50.207949 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:43:50.207949 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:43:50.207949 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:43:50.208751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:43:50.208751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:43:50.208751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:43:50.208751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:43:50.208751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:43:50.208751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:43:50.208751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:43:50.208751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:43:50.208751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:43:50.208751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:43:50.208751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:43:50.208751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:43:50.208751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:43:50.208751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:43:50.208751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:43:50.208751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:43:50.209498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:43:50.209498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:43:50.209498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:43:50.209498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:43:50.209498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:43:50.209498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:43:50.209498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:43:50.209498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:43:50.209498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:43:50.209498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:43:50.209498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:43:50.209498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:43:50.209498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 36/75] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:43:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:43:50.210223 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd6ee5-77c3-4202-af6f-2ff3d08c77fd None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:50.210939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-aa5cb68e-4cec-42ea-bbb8-362220cb8677 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:43:50.210939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:43:50.210939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:43:50.210939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:43:50.210939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:43:50.210939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:43:50.210939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:43:50.210939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:43:50.210939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:43:50.210939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:43:50.210939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:43:50.210939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:43:50.210939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:43:50.210939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:43:50.210939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:43:50.210939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:43:50.211808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:43:50.211808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:43:50.211808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:43:50.211808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:43:50.211808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:43:50.211808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:43:50.211808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:43:50.211808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:43:50.211808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:43:50.211808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:43:50.211808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:43:50.211808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:43:50.211808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:43:50.211808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:43:50.211808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:43:50.211808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:43:50.212549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:43:50.212549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:43:50.212549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:43:50.212549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:43:50.212549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:43:50.212549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:43:50.212549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:43:50.212549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:43:50.212549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:43:50.212549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:43:50.212549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:43:50.212549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:43:50.213970 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 40/76] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:43:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:43:50.214805 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5cb68e-4cec-42ea-bbb8-362220cb8677 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:50.233001 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:50.311478 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd6ee5-77c3-4202-af6f-2ff3d08c77fd None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:50.311716 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5cb68e-4cec-42ea-bbb8-362220cb8677 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:50.325892 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:50.347283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc95ea-ff14-489f-aed1-5c90275657bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:50.347924 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc95ea-ff14-489f-aed1-5c90275657bc None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:50.348206 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc95ea-ff14-489f-aed1-5c90275657bc None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:50.348489 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc95ea-ff14-489f-aed1-5c90275657bc None admin] PATH_INFO: `/v3/users/e3f891d792a74027848d05c2a300e2af` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:50.353302 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4278f3-76e8-4388-b098-00e5437b66a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:50.353918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4278f3-76e8-4388-b098-00e5437b66a3 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:50.354198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4278f3-76e8-4388-b098-00e5437b66a3 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:50.354495 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4278f3-76e8-4388-b098-00e5437b66a3 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:50.355192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4278f3-76e8-4388-b098-00e5437b66a3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:43:50.355539 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4278f3-76e8-4388-b098-00e5437b66a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:50.356936 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4278f3-76e8-4388-b098-00e5437b66a3 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:50.362537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc95ea-ff14-489f-aed1-5c90275657bc None admin] RBAC: Authorizing `identity:get_user(user_id=e3f891d792a74027848d05c2a300e2af)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:50.363941 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc95ea-ff14-489f-aed1-5c90275657bc None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:50.365580 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 41/77] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:43:50 2022] GET /identity/v3/users/e3f891d792a74027848d05c2a300e2af => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:50.370375 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc95ea-ff14-489f-aed1-5c90275657bc None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:50.370509 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 37/78] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:43:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:50.370509 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4278f3-76e8-4388-b098-00e5437b66a3 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:50.446605 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:50.469310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4278f3-76e8-4388-b098-00e5437b66a3 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:50.473976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc95ea-ff14-489f-aed1-5c90275657bc None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:50.504087 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c396b1d-e175-42e0-ae36-f1d3183b3a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:50.504699 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c396b1d-e175-42e0-ae36-f1d3183b3a64 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:50.504977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c396b1d-e175-42e0-ae36-f1d3183b3a64 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:50.505245 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c396b1d-e175-42e0-ae36-f1d3183b3a64 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:50.505906 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c396b1d-e175-42e0-ae36-f1d3183b3a64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:43:50.506209 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c396b1d-e175-42e0-ae36-f1d3183b3a64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:50.507596 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c396b1d-e175-42e0-ae36-f1d3183b3a64 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:50.508870 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8242976-f466-468a-a423-edf283c3b252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:50.509498 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8242976-f466-468a-a423-edf283c3b252 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:50.509781 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8242976-f466-468a-a423-edf283c3b252 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:50.510051 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8242976-f466-468a-a423-edf283c3b252 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:50.511201 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8242976-f466-468a-a423-edf283c3b252 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:50.512636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8242976-f466-468a-a423-edf283c3b252 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:50.514202 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 42/79] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:43:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:50.514988 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8242976-f466-468a-a423-edf283c3b252 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:50.520375 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 38/80] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:43:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:50.520375 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c396b1d-e175-42e0-ae36-f1d3183b3a64 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:50.558516 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:50.615256 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8242976-f466-468a-a423-edf283c3b252 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:50.620463 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c396b1d-e175-42e0-ae36-f1d3183b3a64 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:50.649702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a744eb-2998-4362-9b9a-40b6bde913ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:50.650327 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a744eb-2998-4362-9b9a-40b6bde913ee None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:50.650752 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a744eb-2998-4362-9b9a-40b6bde913ee None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:50.651036 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a744eb-2998-4362-9b9a-40b6bde913ee None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:50.651704 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a744eb-2998-4362-9b9a-40b6bde913ee None admin] RBAC: Adding query filter params (name=member) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:43:50.652009 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a744eb-2998-4362-9b9a-40b6bde913ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:50.653390 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a744eb-2998-4362-9b9a-40b6bde913ee None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:50.655629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f3fb0-a92b-4646-b820-f5214370851a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:50.656530 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f3fb0-a92b-4646-b820-f5214370851a None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:50.656815 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f3fb0-a92b-4646-b820-f5214370851a None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:50.657091 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f3fb0-a92b-4646-b820-f5214370851a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:50.661984 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 43/81] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 31 10:43:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:50.662723 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a744eb-2998-4362-9b9a-40b6bde913ee None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:50.667347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:50.725884 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f3fb0-a92b-4646-b820-f5214370851a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:50.727398 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f3fb0-a92b-4646-b820-f5214370851a None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:50.763467 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a744eb-2998-4362-9b9a-40b6bde913ee None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:50.766523 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:50.797484 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d98327-5221-49f7-a67b-313ba62af26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:50.798121 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d98327-5221-49f7-a67b-313ba62af26d None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:50.798468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d98327-5221-49f7-a67b-313ba62af26d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:50.798752 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d98327-5221-49f7-a67b-313ba62af26d None admin] PATH_INFO: `/v3/users/e3f891d792a74027848d05c2a300e2af` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:50.799940 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d98327-5221-49f7-a67b-313ba62af26d None admin] RBAC: Authorizing `identity:get_user(user_id=e3f891d792a74027848d05c2a300e2af)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:50.801443 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d98327-5221-49f7-a67b-313ba62af26d None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:50.803077 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 44/82] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:43:50 2022] GET /identity/v3/users/e3f891d792a74027848d05c2a300e2af => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:50.803818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d98327-5221-49f7-a67b-313ba62af26d None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:50.829020 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 39/83] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:43:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:43:50.829886 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f3fb0-a92b-4646-b820-f5214370851a None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:50.881102 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:50.912239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d98327-5221-49f7-a67b-313ba62af26d None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:50.938168 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f3fb0-a92b-4646-b820-f5214370851a None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:50.948661 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492de30-7035-433e-8044-803eb0208d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:50.949468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492de30-7035-433e-8044-803eb0208d87 None admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:50.949752 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492de30-7035-433e-8044-803eb0208d87 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:50.950028 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492de30-7035-433e-8044-803eb0208d87 None admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:50.950762 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9492de30-7035-433e-8044-803eb0208d87 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:50.952217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9492de30-7035-433e-8044-803eb0208d87 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:50.971125 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358e6e3-cb87-4424-b561-d597f3320ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:50.971774 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358e6e3-cb87-4424-b561-d597f3320ebe None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:50.972047 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358e6e3-cb87-4424-b561-d597f3320ebe None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:50.972319 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358e6e3-cb87-4424-b561-d597f3320ebe None admin] PATH_INFO: `/v3/users/32a2719366d1460cb5f9b2c1c8e41774` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:50.973468 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f358e6e3-cb87-4424-b561-d597f3320ebe None admin] RBAC: Authorizing `identity:get_user(user_id=32a2719366d1460cb5f9b2c1c8e41774)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:50.974989 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f358e6e3-cb87-4424-b561-d597f3320ebe None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:50.976570 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 40/84] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:43:50 2022] GET /identity/v3/users/32a2719366d1460cb5f9b2c1c8e41774 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:50.977304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358e6e3-cb87-4424-b561-d597f3320ebe None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:50.986610 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:51.050170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 45/85] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 31 10:43:50 2022] POST /identity/v3/users => generated 289 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:43:51.051041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492de30-7035-433e-8044-803eb0208d87 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:51.078050 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358e6e3-cb87-4424-b561-d597f3320ebe None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:51.124065 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05ce7f-86cd-4898-bbcb-e97fe64ae988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:51.124748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05ce7f-86cd-4898-bbcb-e97fe64ae988 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:51.125032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05ce7f-86cd-4898-bbcb-e97fe64ae988 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:51.125296 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05ce7f-86cd-4898-bbcb-e97fe64ae988 None admin] PATH_INFO: `/v3/users/e3f891d792a74027848d05c2a300e2af` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:51.126550 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05ce7f-86cd-4898-bbcb-e97fe64ae988 None admin] RBAC: Authorizing `identity:get_user(user_id=e3f891d792a74027848d05c2a300e2af)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:51.128024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05ce7f-86cd-4898-bbcb-e97fe64ae988 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:51.130445 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 41/86] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:43:50 2022] GET /identity/v3/users/e3f891d792a74027848d05c2a300e2af => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:51.130605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05ce7f-86cd-4898-bbcb-e97fe64ae988 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:51.176305 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492de30-7035-433e-8044-803eb0208d87 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:51.206739 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:51.215374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01539b01-6ed2-4fd2-bd38-ded939a64ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:51.216008 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01539b01-6ed2-4fd2-bd38-ded939a64ee9 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:51.216290 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01539b01-6ed2-4fd2-bd38-ded939a64ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:51.221145 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01539b01-6ed2-4fd2-bd38-ded939a64ee9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:51.227083 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:51.257681 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05ce7f-86cd-4898-bbcb-e97fe64ae988 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:51.300121 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe07962-67e3-4ac5-95a9-1782f5bf8598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:51.301049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe07962-67e3-4ac5-95a9-1782f5bf8598 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:51.301331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe07962-67e3-4ac5-95a9-1782f5bf8598 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:51.301602 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe07962-67e3-4ac5-95a9-1782f5bf8598 None admin] PATH_INFO: `/v3/projects/8d08421140694791b18a512ea16981a6` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:51.302804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe07962-67e3-4ac5-95a9-1782f5bf8598 None admin] RBAC: Authorizing `identity:get_project(project_id=8d08421140694791b18a512ea16981a6)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:51.303239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01539b01-6ed2-4fd2-bd38-ded939a64ee9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:51.304366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe07962-67e3-4ac5-95a9-1782f5bf8598 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:51.305085 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01539b01-6ed2-4fd2-bd38-ded939a64ee9 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:51.306021 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 42/87] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:43:51 2022] GET /identity/v3/projects/8d08421140694791b18a512ea16981a6 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:51.306816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe07962-67e3-4ac5-95a9-1782f5bf8598 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:51.444641 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe07962-67e3-4ac5-95a9-1782f5bf8598 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:51.450261 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 46/88] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:43:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:43:51.451147 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01539b01-6ed2-4fd2-bd38-ded939a64ee9 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:51.486969 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755e522-1b7b-443f-848a-edd825d325ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:51.487602 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755e522-1b7b-443f-848a-edd825d325ea None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:51.487956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755e522-1b7b-443f-848a-edd825d325ea None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:51.488242 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755e522-1b7b-443f-848a-edd825d325ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:51.489814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:51.504212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:51.575670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d755e522-1b7b-443f-848a-edd825d325ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:51.577431 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d755e522-1b7b-443f-848a-edd825d325ea None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:51.578625 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01539b01-6ed2-4fd2-bd38-ded939a64ee9 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:51.620472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa9c8f-417e-4466-9fb4-3c288a469da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:51.621740 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa9c8f-417e-4466-9fb4-3c288a469da1 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:51.622173 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa9c8f-417e-4466-9fb4-3c288a469da1 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:51.622636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa9c8f-417e-4466-9fb4-3c288a469da1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:51.704658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 43/89] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:43:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:43:51.705867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755e522-1b7b-443f-848a-edd825d325ea None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:51.714028 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaa9c8f-417e-4466-9fb4-3c288a469da1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:51.715802 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaa9c8f-417e-4466-9fb4-3c288a469da1 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:51.744424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:51.758102 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:51.819249 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 47/90] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:43:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:43:51.820102 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa9c8f-417e-4466-9fb4-3c288a469da1 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:51.824705 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755e522-1b7b-443f-848a-edd825d325ea None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:51.864702 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d74f8-c58e-4d74-9ac3-8f5649efd2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:51.865635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d74f8-c58e-4d74-9ac3-8f5649efd2d9 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:51.866092 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d74f8-c58e-4d74-9ac3-8f5649efd2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:51.866571 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d74f8-c58e-4d74-9ac3-8f5649efd2d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:51.867613 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d74f8-c58e-4d74-9ac3-8f5649efd2d9 None admin] RBAC: Adding query filter params (role.id=2278da3c52644a819d210b5f9c20b6c1, scope.project.id=8d08421140694791b18a512ea16981a6, user.id=e3f891d792a74027848d05c2a300e2af) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:43:51.868202 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d74f8-c58e-4d74-9ac3-8f5649efd2d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:51.869823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d74f8-c58e-4d74-9ac3-8f5649efd2d9 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:51.877680 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 44/91] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:43:51 2022] GET /identity/v3/role_assignments?user.id=e3f891d792a74027848d05c2a300e2af&scope.project.id=8d08421140694791b18a512ea16981a6&role.id=2278da3c52644a819d210b5f9c20b6c1 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:51.878820 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d74f8-c58e-4d74-9ac3-8f5649efd2d9 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:51.960845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa9c8f-417e-4466-9fb4-3c288a469da1 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:51.994676 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d74f8-c58e-4d74-9ac3-8f5649efd2d9 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:52.003530 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:52.009271 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:52.023340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974fa96c-fe62-4dfd-8d3f-d2effdca01f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:52.024023 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fa96c-fe62-4dfd-8d3f-d2effdca01f1 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:52.024300 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fa96c-fe62-4dfd-8d3f-d2effdca01f1 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:52.024619 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fa96c-fe62-4dfd-8d3f-d2effdca01f1 None admin] PATH_INFO: `/v3/projects/8d08421140694791b18a512ea16981a6` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:52.029866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974fa96c-fe62-4dfd-8d3f-d2effdca01f1 None admin] RBAC: Authorizing `identity:get_project(project_id=8d08421140694791b18a512ea16981a6)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:52.031522 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974fa96c-fe62-4dfd-8d3f-d2effdca01f1 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:52.033258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 48/92] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:43:51 2022] GET /identity/v3/projects/8d08421140694791b18a512ea16981a6 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:52.034023 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974fa96c-fe62-4dfd-8d3f-d2effdca01f1 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:52.038734 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7053532-3a3a-4c9e-8827-2233e7fcb788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:52.039646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7053532-3a3a-4c9e-8827-2233e7fcb788 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:52.040529 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7053532-3a3a-4c9e-8827-2233e7fcb788 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:52.041162 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7053532-3a3a-4c9e-8827-2233e7fcb788 None admin] PATH_INFO: `/v3/projects/8d08421140694791b18a512ea16981a6` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:52.042853 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7053532-3a3a-4c9e-8827-2233e7fcb788 None admin] RBAC: Authorizing `identity:get_project(project_id=8d08421140694791b18a512ea16981a6)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:52.044504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7053532-3a3a-4c9e-8827-2233e7fcb788 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:52.046930 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 45/93] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:43:51 2022] GET /identity/v3/projects/8d08421140694791b18a512ea16981a6 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:52.048207 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7053532-3a3a-4c9e-8827-2233e7fcb788 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:52.174184 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7053532-3a3a-4c9e-8827-2233e7fcb788 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:52.211404 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974fa96c-fe62-4dfd-8d3f-d2effdca01f1 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:52.213218 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c146c-2ef8-45b6-ac2b-07a059b7cf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:52.213946 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c146c-2ef8-45b6-ac2b-07a059b7cf13 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:52.214233 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c146c-2ef8-45b6-ac2b-07a059b7cf13 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:52.214535 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c146c-2ef8-45b6-ac2b-07a059b7cf13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:52.215277 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268c146c-2ef8-45b6-ac2b-07a059b7cf13 None admin] RBAC: Adding query filter params (role.id=31d4b1dc082c41f7b6f18c338e7b51df, scope.project.id=8d08421140694791b18a512ea16981a6, user.id=e3f891d792a74027848d05c2a300e2af) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:43:52.215590 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268c146c-2ef8-45b6-ac2b-07a059b7cf13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:52.217045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268c146c-2ef8-45b6-ac2b-07a059b7cf13 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:52.223986 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 46/94] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:43:52 2022] GET /identity/v3/role_assignments?user.id=e3f891d792a74027848d05c2a300e2af&scope.project.id=8d08421140694791b18a512ea16981a6&role.id=31d4b1dc082c41f7b6f18c338e7b51df => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:52.224890 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c146c-2ef8-45b6-ac2b-07a059b7cf13 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:52.253521 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafb463-f3b4-4d96-b6d6-1507e660e5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:52.254169 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafb463-f3b4-4d96-b6d6-1507e660e5dc None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:52.254478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafb463-f3b4-4d96-b6d6-1507e660e5dc None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:52.254754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafb463-f3b4-4d96-b6d6-1507e660e5dc None admin] PATH_INFO: `/v3/projects/e729141399814f63857cb776be5c236c` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:52.256230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafb463-f3b4-4d96-b6d6-1507e660e5dc None admin] RBAC: Authorizing `identity:get_project(project_id=e729141399814f63857cb776be5c236c)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:52.257768 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafb463-f3b4-4d96-b6d6-1507e660e5dc None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:52.260277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 49/95] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:43:52 2022] GET /identity/v3/projects/e729141399814f63857cb776be5c236c => generated 312 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:52.267671 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafb463-f3b4-4d96-b6d6-1507e660e5dc None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:52.274759 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:52.346995 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c146c-2ef8-45b6-ac2b-07a059b7cf13 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:52.368304 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:52.387625 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32598dab-ae56-4581-b1f4-ca7a3ce99c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:52.388257 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32598dab-ae56-4581-b1f4-ca7a3ce99c57 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:52.388542 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32598dab-ae56-4581-b1f4-ca7a3ce99c57 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:52.388973 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32598dab-ae56-4581-b1f4-ca7a3ce99c57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:52.389729 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32598dab-ae56-4581-b1f4-ca7a3ce99c57 None admin] RBAC: Adding query filter params (role.id=13ca33d694c645d6920930d39dc52a0d, scope.project.id=8d08421140694791b18a512ea16981a6, user.id=32a2719366d1460cb5f9b2c1c8e41774) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:43:52.390743 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32598dab-ae56-4581-b1f4-ca7a3ce99c57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:52.392386 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32598dab-ae56-4581-b1f4-ca7a3ce99c57 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:52.399071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 47/96] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:43:52 2022] GET /identity/v3/role_assignments?user.id=32a2719366d1460cb5f9b2c1c8e41774&scope.project.id=8d08421140694791b18a512ea16981a6&role.id=13ca33d694c645d6920930d39dc52a0d => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:52.436617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafb463-f3b4-4d96-b6d6-1507e660e5dc None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:52.496445 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248faa7-e759-4e63-a4a0-f057aa17ddbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:52.497084 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248faa7-e759-4e63-a4a0-f057aa17ddbb None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:52.497366 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248faa7-e759-4e63-a4a0-f057aa17ddbb None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:52.497635 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248faa7-e759-4e63-a4a0-f057aa17ddbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:52.498410 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0248faa7-e759-4e63-a4a0-f057aa17ddbb None admin] RBAC: Adding query filter params (role.id=2278da3c52644a819d210b5f9c20b6c1, scope.project.id=e729141399814f63857cb776be5c236c, user.id=e3f891d792a74027848d05c2a300e2af) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:43:52.498730 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0248faa7-e759-4e63-a4a0-f057aa17ddbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:52.500761 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0248faa7-e759-4e63-a4a0-f057aa17ddbb None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:52.508823 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 50/97] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:43:52 2022] GET /identity/v3/role_assignments?user.id=e3f891d792a74027848d05c2a300e2af&scope.project.id=e729141399814f63857cb776be5c236c&role.id=2278da3c52644a819d210b5f9c20b6c1 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:53.650318 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254104eb-f327-4b6b-bf9f-65be6e74e6ee None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:53.650845 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254104eb-f327-4b6b-bf9f-65be6e74e6ee None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:53.650916 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254104eb-f327-4b6b-bf9f-65be6e74e6ee None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:53.652655 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 48/98] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:43:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:43:53.658099 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6d53e-86e9-4f40-98d2-8b36f5d71d47 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:53.658567 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6d53e-86e9-4f40-98d2-8b36f5d71d47 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:53.658742 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6d53e-86e9-4f40-98d2-8b36f5d71d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:53.688028 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-f3b6d53e-86e9-4f40-98d2-8b36f5d71d47 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:43:53.689388 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-f3b6d53e-86e9-4f40-98d2-8b36f5d71d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:43:53.743578 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921fbfa-94fd-40dc-b869-3a829a361ec0 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:53.743900 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921fbfa-94fd-40dc-b869-3a829a361ec0 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:53.744133 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921fbfa-94fd-40dc-b869-3a829a361ec0 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:53.745485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 49/99] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:43:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:43:53.751646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6c868-879a-41c2-a907-6f41bcb7d37f None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:53.751937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6c868-879a-41c2-a907-6f41bcb7d37f None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:53.752213 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6c868-879a-41c2-a907-6f41bcb7d37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:53.777606 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-1dd6c868-879a-41c2-a907-6f41bcb7d37f None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:43:53.778851 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-1dd6c868-879a-41c2-a907-6f41bcb7d37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:43:53.817757 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 51/100] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:43:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:43:53.874857 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 50/101] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:43:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:43:54.164513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6d53e-86e9-4f40-98d2-8b36f5d71d47 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:54.242657 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:54.291691 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6d53e-86e9-4f40-98d2-8b36f5d71d47 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:54.327017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd845b-8ea3-448b-bc9d-33744212b1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:54.328189 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd845b-8ea3-448b-bc9d-33744212b1a5 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:54.328474 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd845b-8ea3-448b-bc9d-33744212b1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:54.328754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd845b-8ea3-448b-bc9d-33744212b1a5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:54.339588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd845b-8ea3-448b-bc9d-33744212b1a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:43:54.345496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-6ccd845b-8ea3-448b-bc9d-33744212b1a5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:43:54.345496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:43:54.345496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:43:54.345496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:43:54.345496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:43:54.345496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:43:54.345496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:43:54.345496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:43:54.345496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:43:54.345496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:43:54.345496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:43:54.345496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:43:54.345496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:43:54.345496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:43:54.345496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:43:54.345496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:43:54.346344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:43:54.346344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:43:54.346344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:43:54.346344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:43:54.346344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:43:54.346344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:43:54.346344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:43:54.346344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:43:54.346344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:43:54.346344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:43:54.346344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:43:54.346344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:43:54.346344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:43:54.346344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:43:54.346344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:43:54.346344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:43:54.347216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:43:54.347216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:43:54.347216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:43:54.347216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:43:54.347216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:43:54.347216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:43:54.347216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:43:54.347216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:43:54.347216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:43:54.347216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:43:54.347216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:43:54.347216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:43:54.347216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 52/102] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:43:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:43:54.354075 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6c868-879a-41c2-a907-6f41bcb7d37f None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:54.390849 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:54.409206 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd845b-8ea3-448b-bc9d-33744212b1a5 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:54.466894 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6c868-879a-41c2-a907-6f41bcb7d37f None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:54.509907 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4596b83-4219-4c60-a35b-36f7f2d5ec0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:54.510583 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4596b83-4219-4c60-a35b-36f7f2d5ec0b None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:54.510872 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4596b83-4219-4c60-a35b-36f7f2d5ec0b None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:54.511491 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4596b83-4219-4c60-a35b-36f7f2d5ec0b None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:54.512207 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4596b83-4219-4c60-a35b-36f7f2d5ec0b None admin] RBAC: Adding query filter params (name=reader) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:43:54.512524 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4596b83-4219-4c60-a35b-36f7f2d5ec0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:54.513931 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4596b83-4219-4c60-a35b-36f7f2d5ec0b None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:54.528509 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 51/103] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 31 10:43:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:54.531530 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4596b83-4219-4c60-a35b-36f7f2d5ec0b None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:54.640386 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:54.652083 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4596b83-4219-4c60-a35b-36f7f2d5ec0b None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:54.660595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd845b-8ea3-448b-bc9d-33744212b1a5 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:54.702040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:54.725956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41cdfc-6624-4b35-befb-49a1d28c23f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:54.726630 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41cdfc-6624-4b35-befb-49a1d28c23f2 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:54.726916 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41cdfc-6624-4b35-befb-49a1d28c23f2 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:54.727197 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41cdfc-6624-4b35-befb-49a1d28c23f2 None admin] PATH_INFO: `/v3/users/3790cf1cdc3c4b95afd5eb64ded7322e` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:54.741762 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7dbe1-4200-425b-bbf6-3f2ca963af38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:54.742428 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7dbe1-4200-425b-bbf6-3f2ca963af38 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:54.742727 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7dbe1-4200-425b-bbf6-3f2ca963af38 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:54.743014 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7dbe1-4200-425b-bbf6-3f2ca963af38 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:54.745733 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7dbe1-4200-425b-bbf6-3f2ca963af38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:54.747504 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7dbe1-4200-425b-bbf6-3f2ca963af38 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:54.749128 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 53/104] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:43:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:54.755705 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7dbe1-4200-425b-bbf6-3f2ca963af38 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:54.761949 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d41cdfc-6624-4b35-befb-49a1d28c23f2 None admin] RBAC: Authorizing `identity:get_user(user_id=3790cf1cdc3c4b95afd5eb64ded7322e)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:54.763477 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d41cdfc-6624-4b35-befb-49a1d28c23f2 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:54.765540 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 52/105] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:43:54 2022] GET /identity/v3/users/3790cf1cdc3c4b95afd5eb64ded7322e => generated 289 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:43:54.780917 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41cdfc-6624-4b35-befb-49a1d28c23f2 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:54.899918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41cdfc-6624-4b35-befb-49a1d28c23f2 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:54.942834 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea7c83-dea6-4bc0-83a1-3e582382968d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:54.943845 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea7c83-dea6-4bc0-83a1-3e582382968d None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:54.944415 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea7c83-dea6-4bc0-83a1-3e582382968d None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:54.944846 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea7c83-dea6-4bc0-83a1-3e582382968d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:54.952965 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7dbe1-4200-425b-bbf6-3f2ca963af38 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:54.972892 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:54.990879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507976c4-99c4-4514-b6a3-d5dd7f3c2abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:54.991970 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507976c4-99c4-4514-b6a3-d5dd7f3c2abe None admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:54.992445 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507976c4-99c4-4514-b6a3-d5dd7f3c2abe None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:54.992910 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507976c4-99c4-4514-b6a3-d5dd7f3c2abe None admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:54.993803 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507976c4-99c4-4514-b6a3-d5dd7f3c2abe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:54.995546 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507976c4-99c4-4514-b6a3-d5dd7f3c2abe None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:55.028434 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea7c83-dea6-4bc0-83a1-3e582382968d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:55.029935 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea7c83-dea6-4bc0-83a1-3e582382968d None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:55.120055 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 54/106] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Aug 31 10:43:54 2022] POST /identity/v3/projects => generated 302 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:43:55.121866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251a82d-ef4f-45f8-8723-37429405543f None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:55.122306 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251a82d-ef4f-45f8-8723-37429405543f None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:55.122779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251a82d-ef4f-45f8-8723-37429405543f None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:55.124956 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 55/107] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:43:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:43:55.131933 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56874daf-645a-4ec5-b04e-bf683af527a1 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:55.132447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56874daf-645a-4ec5-b04e-bf683af527a1 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:55.132866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56874daf-645a-4ec5-b04e-bf683af527a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:55.142718 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 53/108] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:43:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 10:43:55.151622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea7c83-dea6-4bc0-83a1-3e582382968d None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:55.183701 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-56874daf-645a-4ec5-b04e-bf683af527a1 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:43:55.184653 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-56874daf-645a-4ec5-b04e-bf683af527a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:43:55.237048 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:55.255528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:55.320190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 56/109] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:43:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:43:55.321750 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051a28f-53b9-4ec7-aa90-a3abd5824a51 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:55.322181 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051a28f-53b9-4ec7-aa90-a3abd5824a51 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:55.322820 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051a28f-53b9-4ec7-aa90-a3abd5824a51 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:55.324294 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 57/110] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:43:55.331169 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea7c83-dea6-4bc0-83a1-3e582382968d None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:55.337743 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d6cd9-108f-4fcc-91a0-15fc5dac310c None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:55.338259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d6cd9-108f-4fcc-91a0-15fc5dac310c None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:55.338771 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d6cd9-108f-4fcc-91a0-15fc5dac310c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:55.379805 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-d89d6cd9-108f-4fcc-91a0-15fc5dac310c None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:43:55.383129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-d89d6cd9-108f-4fcc-91a0-15fc5dac310c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:43:55.408546 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d1777-4472-46d8-b867-5ac806fb584b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:55.409691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d1777-4472-46d8-b867-5ac806fb584b None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:55.409979 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d1777-4472-46d8-b867-5ac806fb584b None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:55.410256 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d1777-4472-46d8-b867-5ac806fb584b None admin] PATH_INFO: `/v3/projects/8d08421140694791b18a512ea16981a6` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:55.413722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d1777-4472-46d8-b867-5ac806fb584b None admin] RBAC: Authorizing `identity:get_project(project_id=8d08421140694791b18a512ea16981a6)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:55.415519 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d1777-4472-46d8-b867-5ac806fb584b None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:55.418060 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 54/111] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:43:55 2022] GET /identity/v3/projects/8d08421140694791b18a512ea16981a6 => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:55.419412 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904692a0-5dc1-4734-ac84-6b89192999ee None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:55.419837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904692a0-5dc1-4734-ac84-6b89192999ee None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:55.420275 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904692a0-5dc1-4734-ac84-6b89192999ee None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:55.422684 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 55/112] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:43:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:43:55.424618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904692a0-5dc1-4734-ac84-6b89192999ee None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:55.491137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 58/113] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:43:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:43:55.493186 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26011974-437b-4604-8ceb-672e138b7ea4 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:55.493729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26011974-437b-4604-8ceb-672e138b7ea4 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:55.498457 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26011974-437b-4604-8ceb-672e138b7ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:55.530054 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-26011974-437b-4604-8ceb-672e138b7ea4 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:43:55.532244 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-26011974-437b-4604-8ceb-672e138b7ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:43:55.568937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904692a0-5dc1-4734-ac84-6b89192999ee None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:55.602806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:55.612038 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed4e7a-e869-4da4-a950-f48fcfae0f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:55.612898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed4e7a-e869-4da4-a950-f48fcfae0f6f None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:55.619350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed4e7a-e869-4da4-a950-f48fcfae0f6f None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:55.619350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed4e7a-e869-4da4-a950-f48fcfae0f6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:55.619350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed4e7a-e869-4da4-a950-f48fcfae0f6f None admin] RBAC: Adding query filter params (role.id=5585e95a315947d1ad314d7978b46e3a, scope.project.id=8d08421140694791b18a512ea16981a6, user.id=3790cf1cdc3c4b95afd5eb64ded7322e) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:43:55.619599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 59/114] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:43:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:43:55.619680 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed4e7a-e869-4da4-a950-f48fcfae0f6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:55.619770 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934e0e4-0762-44c4-9112-ef84b3115eba None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:55.619770 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934e0e4-0762-44c4-9112-ef84b3115eba None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:55.619770 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934e0e4-0762-44c4-9112-ef84b3115eba None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:55.619964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed4e7a-e869-4da4-a950-f48fcfae0f6f None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:55.620038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 60/115] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:43:55.625599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd00090-ecd7-489c-be9b-1faf36631c43 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:55.625921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd00090-ecd7-489c-be9b-1faf36631c43 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:55.626276 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd00090-ecd7-489c-be9b-1faf36631c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:55.634627 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 56/116] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:43:55 2022] GET /identity/v3/role_assignments?user.id=3790cf1cdc3c4b95afd5eb64ded7322e&scope.project.id=8d08421140694791b18a512ea16981a6&role.id=5585e95a315947d1ad314d7978b46e3a => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:43:55.650114 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-1bd00090-ecd7-489c-be9b-1faf36631c43 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:43:55.651391 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-1bd00090-ecd7-489c-be9b-1faf36631c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:43:55.680333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:55.709209 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed4e7a-e869-4da4-a950-f48fcfae0f6f None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:55.731360 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 61/117] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:43:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:43:55.792239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd00090-ecd7-489c-be9b-1faf36631c43 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:55.879830 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed4e7a-e869-4da4-a950-f48fcfae0f6f None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:55.929853 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd00090-ecd7-489c-be9b-1faf36631c43 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:55.939130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60089107-7d3f-4164-8e5f-d1278a9564f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:55.939794 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60089107-7d3f-4164-8e5f-d1278a9564f4 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:55.940079 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60089107-7d3f-4164-8e5f-d1278a9564f4 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:55.940381 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60089107-7d3f-4164-8e5f-d1278a9564f4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:55.962829 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60089107-7d3f-4164-8e5f-d1278a9564f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:43:55.975539 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-60089107-7d3f-4164-8e5f-d1278a9564f4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:43:55.975539 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:43:55.975539 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:43:55.975539 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:43:55.975539 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:43:55.975539 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:43:55.975539 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:43:55.975539 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:43:55.975539 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:43:55.975539 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:43:55.975539 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:43:55.975539 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:43:55.975539 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:43:55.975539 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:43:55.975539 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:43:55.975539 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:43:55.976464 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:43:55.976464 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:43:55.976464 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:43:55.976464 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:43:55.976464 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:43:55.976464 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:43:55.976464 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:43:55.976464 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:43:55.976464 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:43:55.976464 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:43:55.976464 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:43:55.976464 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:43:55.976464 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:43:55.976464 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:43:55.976464 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:43:55.976464 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:43:55.977206 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:43:55.977206 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:43:55.977206 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:43:55.977206 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:43:55.977206 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:43:55.977206 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:43:55.977206 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:43:55.977206 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:43:55.977206 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:43:55.977206 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:43:55.977206 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:43:55.977206 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:43:55.982440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 57/118] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 31 10:43:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:43:55.987995 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d9447-53fd-45e2-8c42-dddb8e766c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:55.988946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d9447-53fd-45e2-8c42-dddb8e766c1e None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:55.990056 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60089107-7d3f-4164-8e5f-d1278a9564f4 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:55.994174 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d9447-53fd-45e2-8c42-dddb8e766c1e None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:55.994778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d9447-53fd-45e2-8c42-dddb8e766c1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:56.021195 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d9447-53fd-45e2-8c42-dddb8e766c1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:43:56.029475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-6a4d9447-53fd-45e2-8c42-dddb8e766c1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:43:56.029475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:43:56.029475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:43:56.029475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:43:56.029475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:43:56.029475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:43:56.029475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:43:56.029475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:43:56.029475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:43:56.029475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:43:56.029475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:43:56.029475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:43:56.029475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:43:56.029475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:43:56.029475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:43:56.029475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:43:56.031041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:43:56.031041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:43:56.031041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:43:56.031041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:43:56.031041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:43:56.031041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:43:56.031041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:43:56.031041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:43:56.031041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:43:56.031041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:43:56.031041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:43:56.031041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:43:56.031041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:43:56.031041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:43:56.031041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:43:56.031041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:43:56.032018 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:43:56.032018 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:43:56.032018 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:43:56.032018 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:43:56.032018 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:43:56.032018 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:43:56.032018 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:43:56.032018 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:43:56.032018 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:43:56.032018 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:43:56.032018 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:43:56.032018 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:43:56.038443 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 62/119] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:43:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:43:56.044447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d9447-53fd-45e2-8c42-dddb8e766c1e None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:56.083010 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:56.091634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:56.171867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60089107-7d3f-4164-8e5f-d1278a9564f4 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:56.210453 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d9447-53fd-45e2-8c42-dddb8e766c1e None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:56.213989 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c39462-fad1-44d3-bd8d-830c93dcafa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:56.214957 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c39462-fad1-44d3-bd8d-830c93dcafa7 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:56.215476 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c39462-fad1-44d3-bd8d-830c93dcafa7 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:56.215987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c39462-fad1-44d3-bd8d-830c93dcafa7 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:56.216897 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c39462-fad1-44d3-bd8d-830c93dcafa7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:43:56.217398 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c39462-fad1-44d3-bd8d-830c93dcafa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:56.220439 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c39462-fad1-44d3-bd8d-830c93dcafa7 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:56.232271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 58/120] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:43:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:56.233422 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c39462-fad1-44d3-bd8d-830c93dcafa7 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:56.254162 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715acd9-6ab7-45c5-952b-2839e3367796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:56.254821 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715acd9-6ab7-45c5-952b-2839e3367796 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:56.255160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715acd9-6ab7-45c5-952b-2839e3367796 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:56.255440 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715acd9-6ab7-45c5-952b-2839e3367796 None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:56.256150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5715acd9-6ab7-45c5-952b-2839e3367796 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:43:56.256741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5715acd9-6ab7-45c5-952b-2839e3367796 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:56.258791 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5715acd9-6ab7-45c5-952b-2839e3367796 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:56.274923 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 63/121] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:43:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:56.276067 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715acd9-6ab7-45c5-952b-2839e3367796 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:56.362786 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c39462-fad1-44d3-bd8d-830c93dcafa7 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:56.370164 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:56.373351 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:56.393275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715acd9-6ab7-45c5-952b-2839e3367796 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:56.398759 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ca5b7-214d-4011-a41e-27322b2149b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:56.399400 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ca5b7-214d-4011-a41e-27322b2149b9 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:56.399680 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ca5b7-214d-4011-a41e-27322b2149b9 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:56.399986 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ca5b7-214d-4011-a41e-27322b2149b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:56.412113 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ca5b7-214d-4011-a41e-27322b2149b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:43:56.417429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-6c7ca5b7-214d-4011-a41e-27322b2149b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:43:56.417429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:43:56.417429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:43:56.417429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:43:56.417429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:43:56.417429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:43:56.417429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:43:56.417429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:43:56.417429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:43:56.417429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:43:56.417429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:43:56.417429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:43:56.417429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:43:56.417429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:43:56.417429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:43:56.417429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:43:56.419258 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:43:56.419258 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:43:56.419258 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:43:56.419258 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:43:56.419258 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:43:56.419258 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:43:56.419258 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:43:56.419258 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:43:56.419258 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:43:56.419258 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:43:56.419258 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:43:56.419258 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:43:56.419258 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:43:56.419258 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:43:56.419258 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:43:56.419258 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:43:56.419997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:43:56.419997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:43:56.419997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:43:56.419997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:43:56.419997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:43:56.419997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:43:56.419997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:43:56.419997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:43:56.419997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:43:56.419997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:43:56.419997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:43:56.419997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:43:56.419997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 59/122] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:43:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:43:56.419997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ca5b7-214d-4011-a41e-27322b2149b9 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:56.437507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca35dd-84bf-45f6-a3aa-c1400c1e39dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:56.438504 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca35dd-84bf-45f6-a3aa-c1400c1e39dc None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:56.438946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca35dd-84bf-45f6-a3aa-c1400c1e39dc None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:56.439366 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca35dd-84bf-45f6-a3aa-c1400c1e39dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:56.450776 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca35dd-84bf-45f6-a3aa-c1400c1e39dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:43:56.456720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-0dca35dd-84bf-45f6-a3aa-c1400c1e39dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:43:56.456720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:43:56.456720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:43:56.456720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:43:56.456720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:43:56.456720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:43:56.456720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:43:56.456720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:43:56.456720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:43:56.456720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:43:56.456720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:43:56.456720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:43:56.456720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:43:56.456720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:43:56.456720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:43:56.456720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:43:56.457620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:43:56.457620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:43:56.457620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:43:56.457620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:43:56.457620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:43:56.457620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:43:56.457620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:43:56.457620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:43:56.457620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:43:56.457620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:43:56.457620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:43:56.457620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:43:56.457620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:43:56.457620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:43:56.457620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:43:56.457620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:43:56.458417 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:43:56.458417 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:43:56.458417 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:43:56.458417 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:43:56.458417 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:43:56.458417 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:43:56.458417 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:43:56.458417 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:43:56.458417 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:43:56.458417 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:43:56.458417 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:43:56.458417 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:43:56.460420 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 64/123] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:43:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:43:56.461393 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca35dd-84bf-45f6-a3aa-c1400c1e39dc None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:56.534456 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ca5b7-214d-4011-a41e-27322b2149b9 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:56.569136 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0d2d7-2d24-421c-8756-18bd799a5242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:56.569819 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0d2d7-2d24-421c-8756-18bd799a5242 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:56.570102 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0d2d7-2d24-421c-8756-18bd799a5242 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:56.570374 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0d2d7-2d24-421c-8756-18bd799a5242 None admin] PATH_INFO: `/v3/users/e3f891d792a74027848d05c2a300e2af` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:56.571551 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b0d2d7-2d24-421c-8756-18bd799a5242 None admin] RBAC: Authorizing `identity:get_user(user_id=e3f891d792a74027848d05c2a300e2af)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:56.571817 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca35dd-84bf-45f6-a3aa-c1400c1e39dc None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:56.573063 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b0d2d7-2d24-421c-8756-18bd799a5242 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:56.575314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 60/124] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:43:56 2022] GET /identity/v3/users/e3f891d792a74027848d05c2a300e2af => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:56.575645 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0d2d7-2d24-421c-8756-18bd799a5242 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:56.611530 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b298993a-75df-46b7-9b03-6c5c7347c044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:56.612626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298993a-75df-46b7-9b03-6c5c7347c044 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:56.613155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298993a-75df-46b7-9b03-6c5c7347c044 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:56.613619 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298993a-75df-46b7-9b03-6c5c7347c044 None admin] PATH_INFO: `/v3/users/32a2719366d1460cb5f9b2c1c8e41774` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:56.614970 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b298993a-75df-46b7-9b03-6c5c7347c044 None admin] RBAC: Authorizing `identity:get_user(user_id=32a2719366d1460cb5f9b2c1c8e41774)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:56.615688 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:56.617864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b298993a-75df-46b7-9b03-6c5c7347c044 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:56.619708 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 65/125] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:43:56 2022] GET /identity/v3/users/32a2719366d1460cb5f9b2c1c8e41774 => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:56.620737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b298993a-75df-46b7-9b03-6c5c7347c044 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:56.633283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:56.690250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0d2d7-2d24-421c-8756-18bd799a5242 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:56.728253 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20aab0c-5bc2-4de8-a21e-fe55e16b2401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:56.729942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20aab0c-5bc2-4de8-a21e-fe55e16b2401 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:56.730593 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20aab0c-5bc2-4de8-a21e-fe55e16b2401 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:56.731083 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20aab0c-5bc2-4de8-a21e-fe55e16b2401 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:56.732155 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20aab0c-5bc2-4de8-a21e-fe55e16b2401 None admin] RBAC: Adding query filter params (name=member) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:43:56.732671 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20aab0c-5bc2-4de8-a21e-fe55e16b2401 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:56.732975 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b298993a-75df-46b7-9b03-6c5c7347c044 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:56.735700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20aab0c-5bc2-4de8-a21e-fe55e16b2401 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:56.745891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 61/126] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 31 10:43:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:56.747330 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20aab0c-5bc2-4de8-a21e-fe55e16b2401 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:56.770658 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f4d595-b9b9-4847-b44a-f7306310ed04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:56.771270 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4d595-b9b9-4847-b44a-f7306310ed04 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:56.771550 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4d595-b9b9-4847-b44a-f7306310ed04 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:56.771877 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4d595-b9b9-4847-b44a-f7306310ed04 None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:56.772546 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f4d595-b9b9-4847-b44a-f7306310ed04 None admin] RBAC: Adding query filter params (name=member) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:43:56.772849 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f4d595-b9b9-4847-b44a-f7306310ed04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:56.774324 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f4d595-b9b9-4847-b44a-f7306310ed04 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:56.788511 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 66/127] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 31 10:43:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:56.788511 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f4d595-b9b9-4847-b44a-f7306310ed04 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:56.857001 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:56.871809 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20aab0c-5bc2-4de8-a21e-fe55e16b2401 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:56.872009 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:56.902599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910a4be-ab10-4725-a104-fd1a931788a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:56.904054 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910a4be-ab10-4725-a104-fd1a931788a8 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:56.904054 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910a4be-ab10-4725-a104-fd1a931788a8 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:56.904054 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910a4be-ab10-4725-a104-fd1a931788a8 None admin] PATH_INFO: `/v3/projects/8d08421140694791b18a512ea16981a6` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:56.905228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3910a4be-ab10-4725-a104-fd1a931788a8 None admin] RBAC: Authorizing `identity:get_project(project_id=8d08421140694791b18a512ea16981a6)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:56.907431 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3910a4be-ab10-4725-a104-fd1a931788a8 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:56.909367 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 62/128] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:43:56 2022] GET /identity/v3/projects/8d08421140694791b18a512ea16981a6 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:56.910249 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910a4be-ab10-4725-a104-fd1a931788a8 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:56.922024 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f4d595-b9b9-4847-b44a-f7306310ed04 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:56.987019 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb0c10-443b-4b22-b27c-305196ccd772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:56.987912 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb0c10-443b-4b22-b27c-305196ccd772 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:56.988506 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb0c10-443b-4b22-b27c-305196ccd772 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:56.988918 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb0c10-443b-4b22-b27c-305196ccd772 None admin] PATH_INFO: `/v3/projects/8d08421140694791b18a512ea16981a6` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:56.991584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb0c10-443b-4b22-b27c-305196ccd772 None admin] RBAC: Authorizing `identity:get_project(project_id=8d08421140694791b18a512ea16981a6)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:56.994995 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb0c10-443b-4b22-b27c-305196ccd772 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:56.997774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 67/129] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:43:56 2022] GET /identity/v3/projects/8d08421140694791b18a512ea16981a6 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:56.999534 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb0c10-443b-4b22-b27c-305196ccd772 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:57.045858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910a4be-ab10-4725-a104-fd1a931788a8 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:57.102137 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc78700-901c-449f-8562-9554194f463a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:57.102833 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc78700-901c-449f-8562-9554194f463a None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:57.103118 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc78700-901c-449f-8562-9554194f463a None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:57.103398 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc78700-901c-449f-8562-9554194f463a None admin] PATH_INFO: `/v3/users/e3f891d792a74027848d05c2a300e2af` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:57.104572 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc78700-901c-449f-8562-9554194f463a None admin] RBAC: Authorizing `identity:get_user(user_id=e3f891d792a74027848d05c2a300e2af)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:57.106123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc78700-901c-449f-8562-9554194f463a None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:57.114426 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 63/130] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:43:56 2022] GET /identity/v3/users/e3f891d792a74027848d05c2a300e2af => generated 275 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:57.114426 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc78700-901c-449f-8562-9554194f463a None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:57.129783 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb0c10-443b-4b22-b27c-305196ccd772 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:57.151535 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:57.172540 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee954a-02d1-4473-91bc-b8fbc5c053a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:57.173586 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee954a-02d1-4473-91bc-b8fbc5c053a6 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:57.174071 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee954a-02d1-4473-91bc-b8fbc5c053a6 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:57.174603 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee954a-02d1-4473-91bc-b8fbc5c053a6 None admin] PATH_INFO: `/v3/users/e3f891d792a74027848d05c2a300e2af` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:57.176008 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee954a-02d1-4473-91bc-b8fbc5c053a6 None admin] RBAC: Authorizing `identity:get_user(user_id=e3f891d792a74027848d05c2a300e2af)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:57.177908 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee954a-02d1-4473-91bc-b8fbc5c053a6 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:57.179939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 68/131] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:43:56 2022] GET /identity/v3/users/e3f891d792a74027848d05c2a300e2af => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:57.185588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee954a-02d1-4473-91bc-b8fbc5c053a6 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:57.196000 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:57.239810 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc78700-901c-449f-8562-9554194f463a None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:57.286739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1aa0bb-22fc-4b65-9240-22b1001618a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:57.287490 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1aa0bb-22fc-4b65-9240-22b1001618a4 None admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:57.287776 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1aa0bb-22fc-4b65-9240-22b1001618a4 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:57.288214 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1aa0bb-22fc-4b65-9240-22b1001618a4 None admin] PATH_INFO: `/v3/projects/8d08421140694791b18a512ea16981a6/users/e3f891d792a74027848d05c2a300e2af/roles/31d4b1dc082c41f7b6f18c338e7b51df` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:57.290930 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1aa0bb-22fc-4b65-9240-22b1001618a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d08421140694791b18a512ea16981a6, user_id=e3f891d792a74027848d05c2a300e2af, role_id=31d4b1dc082c41f7b6f18c338e7b51df)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:57.292396 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1aa0bb-22fc-4b65-9240-22b1001618a4 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:57.303871 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee954a-02d1-4473-91bc-b8fbc5c053a6 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:57.341212 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a8278-bf8e-43e8-8dae-5ef98c9d6141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:57.342271 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a8278-bf8e-43e8-8dae-5ef98c9d6141 None admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:57.342920 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a8278-bf8e-43e8-8dae-5ef98c9d6141 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:57.343895 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a8278-bf8e-43e8-8dae-5ef98c9d6141 None admin] PATH_INFO: `/v3/projects/8d08421140694791b18a512ea16981a6/users/32a2719366d1460cb5f9b2c1c8e41774/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:57.346056 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6a8278-bf8e-43e8-8dae-5ef98c9d6141 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d08421140694791b18a512ea16981a6, user_id=32a2719366d1460cb5f9b2c1c8e41774, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:57.347777 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6a8278-bf8e-43e8-8dae-5ef98c9d6141 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:57.382524 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 64/132] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 31 10:43:57 2022] PUT /identity/v3/projects/8d08421140694791b18a512ea16981a6/users/e3f891d792a74027848d05c2a300e2af/roles/31d4b1dc082c41f7b6f18c338e7b51df => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:43:57.382524 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1aa0bb-22fc-4b65-9240-22b1001618a4 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:57.467431 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 69/133] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 31 10:43:57 2022] PUT /identity/v3/projects/8d08421140694791b18a512ea16981a6/users/32a2719366d1460cb5f9b2c1c8e41774/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:43:57.474533 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a8278-bf8e-43e8-8dae-5ef98c9d6141 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:57.494143 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:57.505182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1aa0bb-22fc-4b65-9240-22b1001618a4 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:57.550400 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e08cc5-f180-44b1-bda6-c58228e6c6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:57.551070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e08cc5-f180-44b1-bda6-c58228e6c6e0 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:57.551349 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e08cc5-f180-44b1-bda6-c58228e6c6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:57.551623 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e08cc5-f180-44b1-bda6-c58228e6c6e0 None admin] PATH_INFO: `/v3/projects/8d08421140694791b18a512ea16981a6` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:57.552818 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e08cc5-f180-44b1-bda6-c58228e6c6e0 None admin] RBAC: Authorizing `identity:get_project(project_id=8d08421140694791b18a512ea16981a6)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:57.558421 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e08cc5-f180-44b1-bda6-c58228e6c6e0 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:57.558421 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 65/134] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:43:57 2022] GET /identity/v3/projects/8d08421140694791b18a512ea16981a6 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:57.569092 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e08cc5-f180-44b1-bda6-c58228e6c6e0 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:57.622159 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:57.672582 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a8278-bf8e-43e8-8dae-5ef98c9d6141 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:57.718821 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33bbfcf-5fe7-4716-8613-31aeb987bf42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:57.720063 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33bbfcf-5fe7-4716-8613-31aeb987bf42 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:57.720570 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33bbfcf-5fe7-4716-8613-31aeb987bf42 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:57.721061 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33bbfcf-5fe7-4716-8613-31aeb987bf42 None admin] PATH_INFO: `/v3/projects/e729141399814f63857cb776be5c236c` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:57.722659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33bbfcf-5fe7-4716-8613-31aeb987bf42 None admin] RBAC: Authorizing `identity:get_project(project_id=e729141399814f63857cb776be5c236c)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:57.724961 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33bbfcf-5fe7-4716-8613-31aeb987bf42 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:57.727840 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 70/135] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:43:57 2022] GET /identity/v3/projects/e729141399814f63857cb776be5c236c => generated 312 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:57.735728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33bbfcf-5fe7-4716-8613-31aeb987bf42 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:57.751013 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e08cc5-f180-44b1-bda6-c58228e6c6e0 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:57.808066 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae6cffb-9c72-4e50-b146-5ac3083c05a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:57.808746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6cffb-9c72-4e50-b146-5ac3083c05a3 None admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:57.809026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6cffb-9c72-4e50-b146-5ac3083c05a3 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:57.809303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6cffb-9c72-4e50-b146-5ac3083c05a3 None admin] PATH_INFO: `/v3/projects/8d08421140694791b18a512ea16981a6/users/e3f891d792a74027848d05c2a300e2af/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:57.812127 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae6cffb-9c72-4e50-b146-5ac3083c05a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d08421140694791b18a512ea16981a6, user_id=e3f891d792a74027848d05c2a300e2af, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:57.813569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae6cffb-9c72-4e50-b146-5ac3083c05a3 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:57.884112 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33bbfcf-5fe7-4716-8613-31aeb987bf42 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:57.891873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 66/136] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 31 10:43:57 2022] PUT /identity/v3/projects/8d08421140694791b18a512ea16981a6/users/e3f891d792a74027848d05c2a300e2af/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 10:43:57.893320 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c1a9d-eb37-4fd5-8067-88908e8f10c5 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:57.893680 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c1a9d-eb37-4fd5-8067-88908e8f10c5 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:57.894011 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c1a9d-eb37-4fd5-8067-88908e8f10c5 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:57.896031 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 67/137] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:43:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:43:57.902146 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361b8f8-023f-4b25-ab89-c0e1b74c606f None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:57.902486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361b8f8-023f-4b25-ab89-c0e1b74c606f None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:57.902796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361b8f8-023f-4b25-ab89-c0e1b74c606f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:57.924977 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdbcb8-6b46-43bf-8053-478d5db98717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:57.925853 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdbcb8-6b46-43bf-8053-478d5db98717 None admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:57.926211 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdbcb8-6b46-43bf-8053-478d5db98717 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:57.926520 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdbcb8-6b46-43bf-8053-478d5db98717 None admin] PATH_INFO: `/v3/projects/e729141399814f63857cb776be5c236c/users/e3f891d792a74027848d05c2a300e2af/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:57.926840 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-c361b8f8-023f-4b25-ab89-c0e1b74c606f None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:43:57.928188 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-c361b8f8-023f-4b25-ab89-c0e1b74c606f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:43:57.928630 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdbcb8-6b46-43bf-8053-478d5db98717 None admin] RBAC: Authorizing `identity:create_grant(project_id=e729141399814f63857cb776be5c236c, user_id=e3f891d792a74027848d05c2a300e2af, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:57.930097 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdbcb8-6b46-43bf-8053-478d5db98717 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:57.934016 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:57.951418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:58.009045 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 71/138] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 31 10:43:57 2022] PUT /identity/v3/projects/e729141399814f63857cb776be5c236c/users/e3f891d792a74027848d05c2a300e2af/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 10:43:58.011088 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ecf4d-1548-4893-aca3-d999605d0b69 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:58.013305 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ecf4d-1548-4893-aca3-d999605d0b69 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:58.015158 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ecf4d-1548-4893-aca3-d999605d0b69 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:58.018213 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 72/139] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:43:58 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:43:58.023967 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a68bd-5e8c-45fb-aa9b-3a9d1a45e1d5 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:58.024258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a68bd-5e8c-45fb-aa9b-3a9d1a45e1d5 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:58.024618 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a68bd-5e8c-45fb-aa9b-3a9d1a45e1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:58.033535 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 68/140] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:43:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:43:58.056136 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-245a68bd-5e8c-45fb-aa9b-3a9d1a45e1d5 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:43:58.057543 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-245a68bd-5e8c-45fb-aa9b-3a9d1a45e1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:43:58.162934 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 73/141] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:43:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:43:58.423396 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c361b8f8-023f-4b25-ab89-c0e1b74c606f None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:58.531020 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c361b8f8-023f-4b25-ab89-c0e1b74c606f None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:58.567335 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee1009-7b31-4525-ba6a-7c2042dad40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:58.567978 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee1009-7b31-4525-ba6a-7c2042dad40f None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:58.568279 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee1009-7b31-4525-ba6a-7c2042dad40f None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:58.568580 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee1009-7b31-4525-ba6a-7c2042dad40f None admin] PATH_INFO: `/v3/domains/default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:58.569716 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ee1009-7b31-4525-ba6a-7c2042dad40f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:58.571216 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ee1009-7b31-4525-ba6a-7c2042dad40f None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:58.573100 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 69/142] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:43:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:58.579312 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a68bd-5e8c-45fb-aa9b-3a9d1a45e1d5 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:58.690575 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a68bd-5e8c-45fb-aa9b-3a9d1a45e1d5 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:58.694646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee1009-7b31-4525-ba6a-7c2042dad40f None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:58.729340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:58.741469 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:58.752055 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e67884-b461-4bf9-8fe2-9c95b801706c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:58.752691 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e67884-b461-4bf9-8fe2-9c95b801706c None admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:58.752980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e67884-b461-4bf9-8fe2-9c95b801706c None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:58.753260 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e67884-b461-4bf9-8fe2-9c95b801706c None admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:58.753940 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e67884-b461-4bf9-8fe2-9c95b801706c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:58.755737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e67884-b461-4bf9-8fe2-9c95b801706c None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:58.879721 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee1009-7b31-4525-ba6a-7c2042dad40f None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:58.901525 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 74/143] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 31 10:43:58 2022] POST /identity/v3/users => generated 283 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 10:43:58.939224 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac17015-ee0f-4f92-a605-e12380894d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:58.940240 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac17015-ee0f-4f92-a605-e12380894d9c None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:58.940820 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac17015-ee0f-4f92-a605-e12380894d9c None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:58.941303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac17015-ee0f-4f92-a605-e12380894d9c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:58.958329 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac17015-ee0f-4f92-a605-e12380894d9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:43:58.964069 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-6ac17015-ee0f-4f92-a605-e12380894d9c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:43:58.964069 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:43:58.964069 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:43:58.964069 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:43:58.964069 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:43:58.964069 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:43:58.964069 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:43:58.964069 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:43:58.964069 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:43:58.964069 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:43:58.964069 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:43:58.964069 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:43:58.964069 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:43:58.964069 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:43:58.964069 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:43:58.964069 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:43:58.965045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:43:58.965045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:43:58.965045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:43:58.965045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:43:58.965045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:43:58.965045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:43:58.965045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:43:58.965045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:43:58.965045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:43:58.965045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:43:58.965045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:43:58.965045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:43:58.965045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:43:58.965045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:43:58.965045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:43:58.965045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:43:58.965827 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:43:58.965827 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:43:58.965827 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:43:58.965827 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:43:58.965827 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:43:58.965827 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:43:58.965827 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:43:58.965827 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:43:58.965827 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:43:58.965827 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:43:58.965827 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:43:58.965827 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:43:58.969093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 70/144] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:43:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:43:58.975203 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e67884-b461-4bf9-8fe2-9c95b801706c None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:59.149780 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e67884-b461-4bf9-8fe2-9c95b801706c None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:59.198724 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae36a7b-a168-413a-9d68-05aee3814d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:59.199267 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae36a7b-a168-413a-9d68-05aee3814d59 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:59.199553 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae36a7b-a168-413a-9d68-05aee3814d59 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:59.199833 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae36a7b-a168-413a-9d68-05aee3814d59 None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:59.200512 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae36a7b-a168-413a-9d68-05aee3814d59 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:43:59.200827 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae36a7b-a168-413a-9d68-05aee3814d59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:59.202896 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae36a7b-a168-413a-9d68-05aee3814d59 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:59.218644 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 75/145] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 31 10:43:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:59.224230 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac17015-ee0f-4f92-a605-e12380894d9c None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:59.362033 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:59.389413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac17015-ee0f-4f92-a605-e12380894d9c None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:59.434561 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e205d4e-9f96-44fc-a04b-ffc739eaec59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:59.435209 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e205d4e-9f96-44fc-a04b-ffc739eaec59 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:59.435486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e205d4e-9f96-44fc-a04b-ffc739eaec59 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:59.435760 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e205d4e-9f96-44fc-a04b-ffc739eaec59 None admin] PATH_INFO: `/v3/users/3790cf1cdc3c4b95afd5eb64ded7322e` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:59.443115 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e205d4e-9f96-44fc-a04b-ffc739eaec59 None admin] RBAC: Authorizing `identity:get_user(user_id=3790cf1cdc3c4b95afd5eb64ded7322e)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:59.444579 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e205d4e-9f96-44fc-a04b-ffc739eaec59 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:59.446543 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 71/146] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:43:59 2022] GET /identity/v3/users/3790cf1cdc3c4b95afd5eb64ded7322e => generated 289 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:59.452269 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae36a7b-a168-413a-9d68-05aee3814d59 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:59.554169 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:43:59.651398 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae36a7b-a168-413a-9d68-05aee3814d59 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:59.691826 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bccd8b6-c8b0-4800-a8f4-30dff1ee7984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:59.692470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bccd8b6-c8b0-4800-a8f4-30dff1ee7984 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:59.692754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bccd8b6-c8b0-4800-a8f4-30dff1ee7984 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:59.693028 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bccd8b6-c8b0-4800-a8f4-30dff1ee7984 None admin] PATH_INFO: `/v3/projects/8d08421140694791b18a512ea16981a6` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:59.694190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bccd8b6-c8b0-4800-a8f4-30dff1ee7984 None admin] RBAC: Authorizing `identity:get_project(project_id=8d08421140694791b18a512ea16981a6)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:59.695682 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bccd8b6-c8b0-4800-a8f4-30dff1ee7984 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:59.697351 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 76/147] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:43:59 2022] GET /identity/v3/projects/8d08421140694791b18a512ea16981a6 => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:43:59.704016 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e205d4e-9f96-44fc-a04b-ffc739eaec59 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:43:59.850896 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e205d4e-9f96-44fc-a04b-ffc739eaec59 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:43:59.893032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb8bf0-85db-49e4-977d-7a1d736203d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:43:59.894634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb8bf0-85db-49e4-977d-7a1d736203d2 None admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:43:59.895140 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb8bf0-85db-49e4-977d-7a1d736203d2 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:43:59.895600 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb8bf0-85db-49e4-977d-7a1d736203d2 None admin] PATH_INFO: `/v3/projects/8d08421140694791b18a512ea16981a6/users/3790cf1cdc3c4b95afd5eb64ded7322e/roles/5585e95a315947d1ad314d7978b46e3a` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:43:59.903756 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb8bf0-85db-49e4-977d-7a1d736203d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d08421140694791b18a512ea16981a6, user_id=3790cf1cdc3c4b95afd5eb64ded7322e, role_id=5585e95a315947d1ad314d7978b46e3a)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:43:59.906044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb8bf0-85db-49e4-977d-7a1d736203d2 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:43:59.976478 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 72/148] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 31 10:43:59 2022] PUT /identity/v3/projects/8d08421140694791b18a512ea16981a6/users/3790cf1cdc3c4b95afd5eb64ded7322e/roles/5585e95a315947d1ad314d7978b46e3a => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:44:00.419561 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d4ecd-0aa3-4505-845e-9529d39e3cd9 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:00.420053 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d4ecd-0aa3-4505-845e-9529d39e3cd9 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:00.420053 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d4ecd-0aa3-4505-845e-9529d39e3cd9 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:00.421732 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 77/149] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:00.427237 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5320719-403a-4cf3-9554-d3fa09559bb8 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:00.427517 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5320719-403a-4cf3-9554-d3fa09559bb8 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:00.427789 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5320719-403a-4cf3-9554-d3fa09559bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:00.442799 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:00.455428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-f5320719-403a-4cf3-9554-d3fa09559bb8 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:00.456649 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-f5320719-403a-4cf3-9554-d3fa09559bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:00.535715 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 73/150] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:00.656018 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700e0f8-a2f6-454b-9240-d5a2cdf6a41f None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:00.656327 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700e0f8-a2f6-454b-9240-d5a2cdf6a41f None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:00.656573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700e0f8-a2f6-454b-9240-d5a2cdf6a41f None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:00.658001 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 78/151] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:00.673042 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368aa05-42b4-48c6-9bac-1d78d6cc4473 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:00.673314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368aa05-42b4-48c6-9bac-1d78d6cc4473 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:00.673511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368aa05-42b4-48c6-9bac-1d78d6cc4473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:00.699410 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-f368aa05-42b4-48c6-9bac-1d78d6cc4473 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:00.700667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-f368aa05-42b4-48c6-9bac-1d78d6cc4473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:00.783211 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 74/152] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:00.908737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700e0f8-a2f6-454b-9240-d5a2cdf6a41f None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:01.022615 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700e0f8-a2f6-454b-9240-d5a2cdf6a41f None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:01.035546 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:01.067186 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36bd961-b229-4978-9f32-996d605e5cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:01.068201 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bd961-b229-4978-9f32-996d605e5cab None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:01.068709 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bd961-b229-4978-9f32-996d605e5cab None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:01.069171 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bd961-b229-4978-9f32-996d605e5cab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:01.081079 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36bd961-b229-4978-9f32-996d605e5cab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:01.086457 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-e36bd961-b229-4978-9f32-996d605e5cab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:44:01.086457 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:01.086457 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:01.086457 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:01.086457 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:01.086457 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:01.086457 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:01.086457 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:01.086457 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:01.086457 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:01.086457 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:01.086457 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:01.086457 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:01.086457 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:01.086457 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:01.086457 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:01.087302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:01.087302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:01.087302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:01.087302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:01.087302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:01.087302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:01.087302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:01.087302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:01.087302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:01.087302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:01.087302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:01.087302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:01.087302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:01.087302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:01.087302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:01.087302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:01.088040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:01.088040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:01.088040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:01.088040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:01.088040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:01.088040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:01.088040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:01.088040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:01.088040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:01.088040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:01.088040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:44:01.088040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:01.090609 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 79/153] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 31 10:44:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:01.097093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368aa05-42b4-48c6-9bac-1d78d6cc4473 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:01.188255 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c5a7e-f2c1-4654-af3b-612e884afb12 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:01.188706 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:01.189208 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c5a7e-f2c1-4654-af3b-612e884afb12 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:01.190215 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c5a7e-f2c1-4654-af3b-612e884afb12 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:01.192799 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 80/154] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:44:01.199762 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f7d8d-e9da-498a-85d2-0df3fb8e0c87 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:01.206496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f7d8d-e9da-498a-85d2-0df3fb8e0c87 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:01.206496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f7d8d-e9da-498a-85d2-0df3fb8e0c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:01.245016 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-8d2f7d8d-e9da-498a-85d2-0df3fb8e0c87 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:01.246547 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-8d2f7d8d-e9da-498a-85d2-0df3fb8e0c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:01.261669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368aa05-42b4-48c6-9bac-1d78d6cc4473 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:01.347788 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51712dfb-0b6b-419d-829d-bbb45ab87d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:01.348384 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51712dfb-0b6b-419d-829d-bbb45ab87d4a None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:01.348666 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51712dfb-0b6b-419d-829d-bbb45ab87d4a None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:01.348937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51712dfb-0b6b-419d-829d-bbb45ab87d4a None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:01.349616 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51712dfb-0b6b-419d-829d-bbb45ab87d4a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:01.349918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51712dfb-0b6b-419d-829d-bbb45ab87d4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:01.351325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51712dfb-0b6b-419d-829d-bbb45ab87d4a None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:01.371180 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 75/155] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:44:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:44:01.372551 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d717eb4-055e-430a-bf38-a6ad16da4135 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:01.372840 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d717eb4-055e-430a-bf38-a6ad16da4135 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:01.373115 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d717eb4-055e-430a-bf38-a6ad16da4135 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:01.378671 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 76/156] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:01.388037 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d717eb4-055e-430a-bf38-a6ad16da4135 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:01.400194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 81/157] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:44:01.401842 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b063f-8941-48cc-a6e3-93ca79bb8dbe None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:01.402351 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b063f-8941-48cc-a6e3-93ca79bb8dbe None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:01.406994 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b063f-8941-48cc-a6e3-93ca79bb8dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:01.430068 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-b96b063f-8941-48cc-a6e3-93ca79bb8dbe None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:01.432273 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-b96b063f-8941-48cc-a6e3-93ca79bb8dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:01.540079 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 82/158] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:01.541766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b063f-8941-48cc-a6e3-93ca79bb8dbe None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:01.562418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d717eb4-055e-430a-bf38-a6ad16da4135 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:01.607315 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:01.611997 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:01.625002 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dcaf7b-60ed-4bcf-81bd-d77f6f3dbdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:01.625280 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcaf7b-60ed-4bcf-81bd-d77f6f3dbdfa None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:01.625565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcaf7b-60ed-4bcf-81bd-d77f6f3dbdfa None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:01.625822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcaf7b-60ed-4bcf-81bd-d77f6f3dbdfa None admin] PATH_INFO: `/v3/users/e3f891d792a74027848d05c2a300e2af` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:01.631079 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dcaf7b-60ed-4bcf-81bd-d77f6f3dbdfa None admin] RBAC: Authorizing `identity:get_user(user_id=e3f891d792a74027848d05c2a300e2af)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:01.632846 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dcaf7b-60ed-4bcf-81bd-d77f6f3dbdfa None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:01.637279 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 77/159] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:44:01 2022] GET /identity/v3/users/e3f891d792a74027848d05c2a300e2af => generated 275 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:44:01.640181 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dcaf7b-60ed-4bcf-81bd-d77f6f3dbdfa None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:01.774257 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b063f-8941-48cc-a6e3-93ca79bb8dbe None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:01.799749 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dcaf7b-60ed-4bcf-81bd-d77f6f3dbdfa None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:01.832028 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2d0fa-28ec-45b8-892d-32e67bcdf636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:01.832679 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2d0fa-28ec-45b8-892d-32e67bcdf636 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:01.832968 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2d0fa-28ec-45b8-892d-32e67bcdf636 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:01.833241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2d0fa-28ec-45b8-892d-32e67bcdf636 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:01.855812 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f2d0fa-28ec-45b8-892d-32e67bcdf636 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:01.861032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-52f2d0fa-28ec-45b8-892d-32e67bcdf636 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:44:01.861032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:01.861032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:01.861032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:01.861032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:01.861032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:01.861032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:01.861032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:01.861032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:01.861032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:01.861032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:01.861032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:01.861032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:01.861032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:01.861032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:01.861032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:01.861871 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:01.861871 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:01.861871 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:01.861871 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:01.861871 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:01.861871 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:01.861871 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:01.861871 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:01.861871 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:01.861871 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:01.861871 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:01.861871 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:01.861871 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:01.861871 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:01.861871 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:01.861871 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:01.863159 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:01.863159 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:01.863159 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:01.863159 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:01.863159 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:01.863159 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:01.863159 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:01.863159 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:01.863159 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:01.863159 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:01.863159 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:44:01.863159 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:01.866687 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 83/160] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:44:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:01.868413 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2d0fa-28ec-45b8-892d-32e67bcdf636 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:01.871784 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864212aa-ba35-4027-89fd-7519e6354015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:01.872768 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864212aa-ba35-4027-89fd-7519e6354015 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:01.873238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864212aa-ba35-4027-89fd-7519e6354015 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:01.873707 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864212aa-ba35-4027-89fd-7519e6354015 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:01.998782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:02.004929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864212aa-ba35-4027-89fd-7519e6354015 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:02.006695 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864212aa-ba35-4027-89fd-7519e6354015 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:02.007306 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:02.021296 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2d0fa-28ec-45b8-892d-32e67bcdf636 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:02.080111 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a7b4a-d3dc-499f-ac39-38c620c3f3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:02.080735 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a7b4a-d3dc-499f-ac39-38c620c3f3e9 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:02.081017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a7b4a-d3dc-499f-ac39-38c620c3f3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:02.081315 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a7b4a-d3dc-499f-ac39-38c620c3f3e9 None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:02.083240 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a7b4a-d3dc-499f-ac39-38c620c3f3e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:02.083240 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a7b4a-d3dc-499f-ac39-38c620c3f3e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:02.083705 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a7b4a-d3dc-499f-ac39-38c620c3f3e9 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:02.097329 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 84/161] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:02.098272 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a7b4a-d3dc-499f-ac39-38c620c3f3e9 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:02.180553 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 78/162] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:44:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 10:44:02.182686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864212aa-ba35-4027-89fd-7519e6354015 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:02.278434 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a7b4a-d3dc-499f-ac39-38c620c3f3e9 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:02.337330 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864212aa-ba35-4027-89fd-7519e6354015 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:02.346747 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:02.362887 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4480aa-9fd6-43d4-b78d-c71c48e35452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:02.364989 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4480aa-9fd6-43d4-b78d-c71c48e35452 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:02.365281 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4480aa-9fd6-43d4-b78d-c71c48e35452 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:02.365550 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4480aa-9fd6-43d4-b78d-c71c48e35452 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:02.388688 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4480aa-9fd6-43d4-b78d-c71c48e35452 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:02.394766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-8f4480aa-9fd6-43d4-b78d-c71c48e35452 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:44:02.394766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:02.394766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:02.394766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:02.394766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:02.394766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:02.394766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:02.394766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:02.394766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:02.394766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:02.394766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:02.394766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:02.394766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:02.394766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:02.394766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:02.394766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:02.395713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:02.395713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:02.395713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:02.395713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:02.395713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:02.395713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:02.395713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:02.395713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:02.395713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:02.395713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:02.395713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:02.395713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:02.395713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:02.395713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:02.395713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:02.395713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:02.396477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:02.396477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:02.396477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:02.396477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:02.396477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:02.396477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:02.396477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:02.396477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:02.396477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:02.396477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:02.396477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:44:02.396477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:02.401328 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299c7f7-d740-456f-a532-71f48596f5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:02.402026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299c7f7-d740-456f-a532-71f48596f5d1 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:02.402305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299c7f7-d740-456f-a532-71f48596f5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:02.402601 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299c7f7-d740-456f-a532-71f48596f5d1 None admin] PATH_INFO: `/v3/users/32a2719366d1460cb5f9b2c1c8e41774` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:02.403802 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299c7f7-d740-456f-a532-71f48596f5d1 None admin] RBAC: Authorizing `identity:get_user(user_id=32a2719366d1460cb5f9b2c1c8e41774)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:02.405658 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 85/163] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:02.406639 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4480aa-9fd6-43d4-b78d-c71c48e35452 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:02.407473 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299c7f7-d740-456f-a532-71f48596f5d1 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:02.411922 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 79/164] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:44:02 2022] GET /identity/v3/users/32a2719366d1460cb5f9b2c1c8e41774 => generated 247 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:02.412812 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299c7f7-d740-456f-a532-71f48596f5d1 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:02.437815 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:02.574335 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4480aa-9fd6-43d4-b78d-c71c48e35452 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:02.611864 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299c7f7-d740-456f-a532-71f48596f5d1 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:02.645131 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96b28d-5c9a-4c2b-9a32-70f6b55219a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:02.645765 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96b28d-5c9a-4c2b-9a32-70f6b55219a2 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:02.646038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96b28d-5c9a-4c2b-9a32-70f6b55219a2 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:02.646306 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96b28d-5c9a-4c2b-9a32-70f6b55219a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:02.667643 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f96b28d-5c9a-4c2b-9a32-70f6b55219a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:02.673239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-6f96b28d-5c9a-4c2b-9a32-70f6b55219a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:44:02.673239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:02.673239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:02.673239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:02.673239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:02.673239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:02.673239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:02.673239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:02.673239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:02.673239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:02.673239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:02.673239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:02.673239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:02.673239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:02.673239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:02.673239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:02.674124 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:02.674124 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:02.674124 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:02.674124 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:02.674124 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:02.674124 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:02.674124 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:02.674124 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:02.674124 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:02.674124 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:02.674124 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:02.674124 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:02.674124 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:02.674124 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:02.674124 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:02.674124 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:02.674930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:02.674930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:02.674930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:02.674930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:02.674930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:02.674930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:02.674930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:02.674930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:02.674930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:02.674930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:02.674930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:44:02.674930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:02.674930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 86/165] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:02.675575 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96b28d-5c9a-4c2b-9a32-70f6b55219a2 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:02.691361 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc501668-b93c-43fc-9e28-337db22f3fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:02.692395 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc501668-b93c-43fc-9e28-337db22f3fa8 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:02.692887 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc501668-b93c-43fc-9e28-337db22f3fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:02.693386 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc501668-b93c-43fc-9e28-337db22f3fa8 None admin] PATH_INFO: `/v3/projects/8d08421140694791b18a512ea16981a6` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:02.695545 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc501668-b93c-43fc-9e28-337db22f3fa8 None admin] RBAC: Authorizing `identity:get_project(project_id=8d08421140694791b18a512ea16981a6)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:02.697310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc501668-b93c-43fc-9e28-337db22f3fa8 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:02.699835 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 80/166] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:02 2022] GET /identity/v3/projects/8d08421140694791b18a512ea16981a6 => generated 298 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:02.700982 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc501668-b93c-43fc-9e28-337db22f3fa8 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:02.754299 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:02.783544 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:02.787475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96b28d-5c9a-4c2b-9a32-70f6b55219a2 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:02.826212 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7eddfd-6357-4340-bda1-e17502029958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:02.827359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7eddfd-6357-4340-bda1-e17502029958 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:02.827829 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7eddfd-6357-4340-bda1-e17502029958 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:02.828291 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7eddfd-6357-4340-bda1-e17502029958 None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:02.829205 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7eddfd-6357-4340-bda1-e17502029958 None admin] RBAC: Adding query filter params (name=member) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:02.829697 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7eddfd-6357-4340-bda1-e17502029958 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:02.831315 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7eddfd-6357-4340-bda1-e17502029958 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:02.840702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 87/167] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 31 10:44:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:02.841821 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7eddfd-6357-4340-bda1-e17502029958 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:02.950712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7eddfd-6357-4340-bda1-e17502029958 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:02.953180 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc501668-b93c-43fc-9e28-337db22f3fa8 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:02.988297 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584eba0-4f54-4884-b1fc-92cea106671b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:02.989301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584eba0-4f54-4884-b1fc-92cea106671b None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:02.989783 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584eba0-4f54-4884-b1fc-92cea106671b None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:02.990292 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584eba0-4f54-4884-b1fc-92cea106671b None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:02.991170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3584eba0-4f54-4884-b1fc-92cea106671b None admin] RBAC: Adding query filter params (name=member) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:02.991678 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3584eba0-4f54-4884-b1fc-92cea106671b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:02.993302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3584eba0-4f54-4884-b1fc-92cea106671b None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:02.995043 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:03.004903 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 88/168] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 31 10:44:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:03.006071 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584eba0-4f54-4884-b1fc-92cea106671b None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:03.047739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de963d-1cf9-451b-8a25-e75daf7efb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:03.048372 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de963d-1cf9-451b-8a25-e75daf7efb98 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:03.048646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de963d-1cf9-451b-8a25-e75daf7efb98 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:03.048921 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de963d-1cf9-451b-8a25-e75daf7efb98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:03.121347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584eba0-4f54-4884-b1fc-92cea106671b None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:03.158191 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dccc1f-f061-4a3e-9cbf-cf4ffd40c89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:03.159202 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dccc1f-f061-4a3e-9cbf-cf4ffd40c89e None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:03.159674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dccc1f-f061-4a3e-9cbf-cf4ffd40c89e None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:03.160135 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dccc1f-f061-4a3e-9cbf-cf4ffd40c89e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:03.161045 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dccc1f-f061-4a3e-9cbf-cf4ffd40c89e None admin] RBAC: Adding query filter params (role.id=31d4b1dc082c41f7b6f18c338e7b51df, scope.project.id=8d08421140694791b18a512ea16981a6, user.id=e3f891d792a74027848d05c2a300e2af) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:03.161568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dccc1f-f061-4a3e-9cbf-cf4ffd40c89e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:03.163294 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dccc1f-f061-4a3e-9cbf-cf4ffd40c89e None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:03.170040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 89/169] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:44:03 2022] GET /identity/v3/role_assignments?user.id=e3f891d792a74027848d05c2a300e2af&scope.project.id=8d08421140694791b18a512ea16981a6&role.id=31d4b1dc082c41f7b6f18c338e7b51df => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:03.171224 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dccc1f-f061-4a3e-9cbf-cf4ffd40c89e None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:03.200673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de963d-1cf9-451b-8a25-e75daf7efb98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:03.203160 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de963d-1cf9-451b-8a25-e75daf7efb98 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:03.240472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:03.276213 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:03.303104 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dccc1f-f061-4a3e-9cbf-cf4ffd40c89e None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:03.377684 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356cc9b9-636e-43fb-9ba0-b69a7142dab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:03.378645 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356cc9b9-636e-43fb-9ba0-b69a7142dab8 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:03.379175 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356cc9b9-636e-43fb-9ba0-b69a7142dab8 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:03.379588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356cc9b9-636e-43fb-9ba0-b69a7142dab8 None admin] PATH_INFO: `/v3/users/e3f891d792a74027848d05c2a300e2af` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:03.387460 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356cc9b9-636e-43fb-9ba0-b69a7142dab8 None admin] RBAC: Authorizing `identity:get_user(user_id=e3f891d792a74027848d05c2a300e2af)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:03.389364 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356cc9b9-636e-43fb-9ba0-b69a7142dab8 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:03.392130 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 90/170] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:44:03 2022] GET /identity/v3/users/e3f891d792a74027848d05c2a300e2af => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:03.393166 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356cc9b9-636e-43fb-9ba0-b69a7142dab8 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:03.415762 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 81/171] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:44:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 715 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:44:03.416517 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de963d-1cf9-451b-8a25-e75daf7efb98 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:03.532573 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de963d-1cf9-451b-8a25-e75daf7efb98 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:03.564052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:03.573698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c65b6-4e0f-4f9d-b22f-0e05988fd442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:03.574832 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c65b6-4e0f-4f9d-b22f-0e05988fd442 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:03.579310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c65b6-4e0f-4f9d-b22f-0e05988fd442 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:03.579310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c65b6-4e0f-4f9d-b22f-0e05988fd442 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:03.593722 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356cc9b9-636e-43fb-9ba0-b69a7142dab8 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:03.648441 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:03.665289 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b20692-99ab-4890-ad8e-51113d5667d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:03.665947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b20692-99ab-4890-ad8e-51113d5667d9 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:03.666226 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b20692-99ab-4890-ad8e-51113d5667d9 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:03.666531 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b20692-99ab-4890-ad8e-51113d5667d9 None admin] PATH_INFO: `/v3/users/e3f891d792a74027848d05c2a300e2af` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:03.667754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b20692-99ab-4890-ad8e-51113d5667d9 None admin] RBAC: Authorizing `identity:get_user(user_id=e3f891d792a74027848d05c2a300e2af)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:03.669280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b20692-99ab-4890-ad8e-51113d5667d9 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:03.671525 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 91/172] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:44:03 2022] GET /identity/v3/users/e3f891d792a74027848d05c2a300e2af => generated 275 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:03.675125 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b20692-99ab-4890-ad8e-51113d5667d9 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:03.693352 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4c65b6-4e0f-4f9d-b22f-0e05988fd442 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:03.694894 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4c65b6-4e0f-4f9d-b22f-0e05988fd442 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:03.790790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b20692-99ab-4890-ad8e-51113d5667d9 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:03.809486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 82/173] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:44:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:44:03.814982 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517b1e1-4772-4e8e-b2a8-196a6a4cecd5 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:03.815286 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517b1e1-4772-4e8e-b2a8-196a6a4cecd5 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:03.815571 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517b1e1-4772-4e8e-b2a8-196a6a4cecd5 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:03.817087 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 83/174] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:03.818312 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a54a02-686f-41a4-a602-c0329f2d300d None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:03.818621 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a54a02-686f-41a4-a602-c0329f2d300d None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:03.818898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a54a02-686f-41a4-a602-c0329f2d300d None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:03.825146 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 84/175] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:03.826060 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a54a02-686f-41a4-a602-c0329f2d300d None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:03.831828 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7e368-41ea-4b17-9c14-1dc597bbd1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:03.832472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7e368-41ea-4b17-9c14-1dc597bbd1d5 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:03.832749 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7e368-41ea-4b17-9c14-1dc597bbd1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:03.833018 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7e368-41ea-4b17-9c14-1dc597bbd1d5 None admin] PATH_INFO: `/v3/projects/8d08421140694791b18a512ea16981a6` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:03.834224 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7e368-41ea-4b17-9c14-1dc597bbd1d5 None admin] RBAC: Authorizing `identity:get_project(project_id=8d08421140694791b18a512ea16981a6)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:03.835782 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7e368-41ea-4b17-9c14-1dc597bbd1d5 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:03.837610 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 92/176] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:03 2022] GET /identity/v3/projects/8d08421140694791b18a512ea16981a6 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:03.838598 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb681f-c0a8-4dcc-a605-ef9332c78a2d None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:03.838877 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb681f-c0a8-4dcc-a605-ef9332c78a2d None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:03.839150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb681f-c0a8-4dcc-a605-ef9332c78a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:03.859412 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-1bcb681f-c0a8-4dcc-a605-ef9332c78a2d None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:03.860625 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-1bcb681f-c0a8-4dcc-a605-ef9332c78a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:03.866817 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:03.923445 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:03.931766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 93/177] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:03.934150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2847fa-3793-4983-ba67-8cadd03ad62c None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:03.934150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2847fa-3793-4983-ba67-8cadd03ad62c None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:03.934150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2847fa-3793-4983-ba67-8cadd03ad62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:03.944573 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a54a02-686f-41a4-a602-c0329f2d300d None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:03.951799 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-cf2847fa-3793-4983-ba67-8cadd03ad62c None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:03.955198 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-cf2847fa-3793-4983-ba67-8cadd03ad62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:03.981728 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf00fe-d719-4ef8-bc97-47f951892345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:03.982470 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf00fe-d719-4ef8-bc97-47f951892345 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:03.982750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf00fe-d719-4ef8-bc97-47f951892345 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:03.983653 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf00fe-d719-4ef8-bc97-47f951892345 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:04.031400 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 94/178] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:04.042633 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2847fa-3793-4983-ba67-8cadd03ad62c None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:04.072120 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaf00fe-d719-4ef8-bc97-47f951892345 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:04.073598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaf00fe-d719-4ef8-bc97-47f951892345 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:04.110443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:04.161857 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2847fa-3793-4983-ba67-8cadd03ad62c None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:04.190492 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:04.206143 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75c3cef-19c6-4e06-96b7-c5fdb6956432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:04.206641 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 85/179] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:44:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:44:04.207107 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c3cef-19c6-4e06-96b7-c5fdb6956432 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:04.207206 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c3cef-19c6-4e06-96b7-c5fdb6956432 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:04.207426 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf00fe-d719-4ef8-bc97-47f951892345 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:04.207522 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c3cef-19c6-4e06-96b7-c5fdb6956432 None admin] PATH_INFO: `/v3/projects/e729141399814f63857cb776be5c236c` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:04.208701 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75c3cef-19c6-4e06-96b7-c5fdb6956432 None admin] RBAC: Authorizing `identity:get_project(project_id=e729141399814f63857cb776be5c236c)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:04.210175 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75c3cef-19c6-4e06-96b7-c5fdb6956432 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:04.212000 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 95/180] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:04 2022] GET /identity/v3/projects/e729141399814f63857cb776be5c236c => generated 312 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:04.213275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ddbedf-49a7-4d40-9138-cd91069ea399 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:04.213562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ddbedf-49a7-4d40-9138-cd91069ea399 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:04.213838 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ddbedf-49a7-4d40-9138-cd91069ea399 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:04.215077 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 96/181] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:04.216158 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edc0aa-581c-45bf-a907-58a96ccde77e None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:04.216542 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edc0aa-581c-45bf-a907-58a96ccde77e None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:04.216818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edc0aa-581c-45bf-a907-58a96ccde77e None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:04.218740 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 97/182] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:04.219450 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80edc0aa-581c-45bf-a907-58a96ccde77e None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:04.334459 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80edc0aa-581c-45bf-a907-58a96ccde77e None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:04.347859 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf00fe-d719-4ef8-bc97-47f951892345 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:04.367803 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b01228-8c48-46f6-b7e4-02317582da17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:04.368421 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b01228-8c48-46f6-b7e4-02317582da17 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:04.368694 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b01228-8c48-46f6-b7e4-02317582da17 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:04.368958 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b01228-8c48-46f6-b7e4-02317582da17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:04.376731 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:04.380062 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b01228-8c48-46f6-b7e4-02317582da17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:04.381762 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609e493-63c6-4724-801c-b18a2f805e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:04.382362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609e493-63c6-4724-801c-b18a2f805e84 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:04.382689 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609e493-63c6-4724-801c-b18a2f805e84 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:04.382962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609e493-63c6-4724-801c-b18a2f805e84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:04.383684 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5609e493-63c6-4724-801c-b18a2f805e84 None admin] RBAC: Adding query filter params (role.id=13ca33d694c645d6920930d39dc52a0d, scope.project.id=8d08421140694791b18a512ea16981a6, user.id=32a2719366d1460cb5f9b2c1c8e41774) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:04.383988 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5609e493-63c6-4724-801c-b18a2f805e84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:04.384914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-09b01228-8c48-46f6-b7e4-02317582da17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:44:04.384914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:04.384914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:04.384914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:04.384914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:04.384914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:04.384914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:04.384914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:04.384914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:04.384914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:04.384914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:04.384914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:04.384914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:04.384914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:04.384914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:04.384914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:04.385872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:04.385872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:04.385872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:04.385872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:04.385872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:04.385872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:04.385872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:04.385872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:04.385872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:04.385872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:04.385872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:04.385872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:04.385872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:04.385872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:04.385872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:04.385872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:04.390511 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:04.390511 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:04.390511 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:04.390511 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:04.390511 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:04.390511 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:04.390511 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:04.390511 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:04.390511 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:04.390511 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:04.390511 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:44:04.390511 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:04.391123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5609e493-63c6-4724-801c-b18a2f805e84 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:04.391198 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 98/183] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:44:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:04.391198 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b01228-8c48-46f6-b7e4-02317582da17 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:04.397155 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 86/184] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:44:04 2022] GET /identity/v3/role_assignments?user.id=32a2719366d1460cb5f9b2c1c8e41774&scope.project.id=8d08421140694791b18a512ea16981a6&role.id=13ca33d694c645d6920930d39dc52a0d => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:04.397155 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609e493-63c6-4724-801c-b18a2f805e84 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:04.456103 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:04.507061 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609e493-63c6-4724-801c-b18a2f805e84 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:04.507590 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b01228-8c48-46f6-b7e4-02317582da17 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:04.547579 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c16580-781a-4530-bf08-4f0dc925c952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:04.548214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c16580-781a-4530-bf08-4f0dc925c952 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:04.548519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c16580-781a-4530-bf08-4f0dc925c952 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:04.548796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c16580-781a-4530-bf08-4f0dc925c952 None admin] PATH_INFO: `/v3/projects/8d08421140694791b18a512ea16981a6` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:04.550025 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c16580-781a-4530-bf08-4f0dc925c952 None admin] RBAC: Authorizing `identity:get_project(project_id=8d08421140694791b18a512ea16981a6)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:04.551245 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7522b3-a9ad-49d5-b06a-d5ecc8c079dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:04.551718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c16580-781a-4530-bf08-4f0dc925c952 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:04.551882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7522b3-a9ad-49d5-b06a-d5ecc8c079dc None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:04.552135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7522b3-a9ad-49d5-b06a-d5ecc8c079dc None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:04.552397 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7522b3-a9ad-49d5-b06a-d5ecc8c079dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:04.553128 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7522b3-a9ad-49d5-b06a-d5ecc8c079dc None admin] RBAC: Adding query filter params (role.id=2278da3c52644a819d210b5f9c20b6c1, scope.project.id=e729141399814f63857cb776be5c236c, user.id=e3f891d792a74027848d05c2a300e2af) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:04.553379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 99/185] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:04 2022] GET /identity/v3/projects/8d08421140694791b18a512ea16981a6 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:04.554598 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e20b1-1e27-43d1-b0af-003a3ec1934b None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:04.554876 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e20b1-1e27-43d1-b0af-003a3ec1934b None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:04.555148 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e20b1-1e27-43d1-b0af-003a3ec1934b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:04.558221 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7522b3-a9ad-49d5-b06a-d5ecc8c079dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:04.561595 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7522b3-a9ad-49d5-b06a-d5ecc8c079dc None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:04.566614 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 87/186] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:44:04 2022] GET /identity/v3/role_assignments?user.id=e3f891d792a74027848d05c2a300e2af&scope.project.id=e729141399814f63857cb776be5c236c&role.id=2278da3c52644a819d210b5f9c20b6c1 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:04.575938 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13feea20-5425-4f3d-bf2c-797ecf6dcc40 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:04.576229 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13feea20-5425-4f3d-bf2c-797ecf6dcc40 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:04.576509 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13feea20-5425-4f3d-bf2c-797ecf6dcc40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:04.577186 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-211e20b1-1e27-43d1-b0af-003a3ec1934b None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:04.578443 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-211e20b1-1e27-43d1-b0af-003a3ec1934b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:04.604092 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-13feea20-5425-4f3d-bf2c-797ecf6dcc40 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:04.605732 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-13feea20-5425-4f3d-bf2c-797ecf6dcc40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:04.672117 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 100/187] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:04.679009 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:04.680164 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e20b1-1e27-43d1-b0af-003a3ec1934b None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:04.690634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 88/188] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:04.691687 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13feea20-5425-4f3d-bf2c-797ecf6dcc40 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:04.755294 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:04.839048 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e20b1-1e27-43d1-b0af-003a3ec1934b None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:04.839574 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13feea20-5425-4f3d-bf2c-797ecf6dcc40 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:04.890555 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f54b8-2693-46de-8b9b-049ce36bd694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:04.891192 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f54b8-2693-46de-8b9b-049ce36bd694 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:04.891471 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f54b8-2693-46de-8b9b-049ce36bd694 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:04.891593 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff4ed9-61f1-4bf9-b6d8-2efcc97be45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:04.891772 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f54b8-2693-46de-8b9b-049ce36bd694 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:04.892192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff4ed9-61f1-4bf9-b6d8-2efcc97be45e None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:04.892514 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff4ed9-61f1-4bf9-b6d8-2efcc97be45e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:04.892863 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff4ed9-61f1-4bf9-b6d8-2efcc97be45e None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:04.893565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff4ed9-61f1-4bf9-b6d8-2efcc97be45e None admin] RBAC: Adding query filter params (name=admin) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:04.893918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff4ed9-61f1-4bf9-b6d8-2efcc97be45e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:04.895426 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff4ed9-61f1-4bf9-b6d8-2efcc97be45e None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:04.905814 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 89/189] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:04.906091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f54b8-2693-46de-8b9b-049ce36bd694 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:04.906792 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff4ed9-61f1-4bf9-b6d8-2efcc97be45e None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:04.911991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-a96f54b8-2693-46de-8b9b-049ce36bd694 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:44:04.911991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:04.911991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:04.911991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:04.911991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:04.911991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:04.911991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:04.911991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:04.911991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:04.911991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:04.911991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:04.911991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:04.911991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:04.911991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:04.911991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:04.911991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:04.912880 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:04.912880 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:04.912880 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:04.912880 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:04.912880 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:04.912880 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:04.912880 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:04.912880 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:04.912880 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:04.912880 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:04.912880 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:04.912880 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:04.912880 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:04.912880 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:04.912880 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:04.912880 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:04.913694 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:04.913694 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:04.913694 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:04.913694 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:04.913694 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:04.913694 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:04.913694 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:04.913694 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:04.913694 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:04.913694 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:04.913694 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:44:04.913694 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:04.914354 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 101/190] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 31 10:44:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:04.918928 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f54b8-2693-46de-8b9b-049ce36bd694 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:04.980052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:05.033349 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff4ed9-61f1-4bf9-b6d8-2efcc97be45e None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:05.036406 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f54b8-2693-46de-8b9b-049ce36bd694 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:05.048879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:05.067963 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf51078-ee70-4720-8ccb-e9f33f9a5416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:05.068586 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf51078-ee70-4720-8ccb-e9f33f9a5416 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:05.068845 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf51078-ee70-4720-8ccb-e9f33f9a5416 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:05.069119 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf51078-ee70-4720-8ccb-e9f33f9a5416 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:05.069842 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf51078-ee70-4720-8ccb-e9f33f9a5416 None admin] RBAC: Adding query filter params (role.id=2278da3c52644a819d210b5f9c20b6c1, scope.project.id=8d08421140694791b18a512ea16981a6, user.id=e3f891d792a74027848d05c2a300e2af) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:05.070148 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf51078-ee70-4720-8ccb-e9f33f9a5416 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:05.071617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf51078-ee70-4720-8ccb-e9f33f9a5416 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:05.075423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9c1b8-d1a1-4243-ab07-8ad44ae8d70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:05.076123 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9c1b8-d1a1-4243-ab07-8ad44ae8d70d None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:05.076723 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9c1b8-d1a1-4243-ab07-8ad44ae8d70d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:05.077183 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9c1b8-d1a1-4243-ab07-8ad44ae8d70d None admin] PATH_INFO: `/v3/users/9ede9378f81a45cc90c51b31386054c2` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:05.077548 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 90/191] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:44:04 2022] GET /identity/v3/role_assignments?user.id=e3f891d792a74027848d05c2a300e2af&scope.project.id=8d08421140694791b18a512ea16981a6&role.id=2278da3c52644a819d210b5f9c20b6c1 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:05.077548 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf51078-ee70-4720-8ccb-e9f33f9a5416 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:05.105455 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca9c1b8-d1a1-4243-ab07-8ad44ae8d70d None admin] RBAC: Authorizing `identity:get_user(user_id=9ede9378f81a45cc90c51b31386054c2)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:05.107998 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca9c1b8-d1a1-4243-ab07-8ad44ae8d70d None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:05.112142 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 102/192] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:44:04 2022] GET /identity/v3/users/9ede9378f81a45cc90c51b31386054c2 => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:05.113386 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9c1b8-d1a1-4243-ab07-8ad44ae8d70d None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:05.204108 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf51078-ee70-4720-8ccb-e9f33f9a5416 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:05.222935 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:05.247821 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a90c6-0e4c-4b6b-bb6c-b55eb9626d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:05.248860 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a90c6-0e4c-4b6b-bb6c-b55eb9626d21 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:05.249146 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a90c6-0e4c-4b6b-bb6c-b55eb9626d21 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:05.249421 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a90c6-0e4c-4b6b-bb6c-b55eb9626d21 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:05.250579 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6a90c6-0e4c-4b6b-bb6c-b55eb9626d21 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:05.250579 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6a90c6-0e4c-4b6b-bb6c-b55eb9626d21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:05.251923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6a90c6-0e4c-4b6b-bb6c-b55eb9626d21 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:05.252171 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9c1b8-d1a1-4243-ab07-8ad44ae8d70d None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:05.261494 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 91/193] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:44:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:05.263371 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a90c6-0e4c-4b6b-bb6c-b55eb9626d21 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:05.298306 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031984a6-5359-4977-9655-4ba6185c524f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:05.298987 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031984a6-5359-4977-9655-4ba6185c524f None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:05.299267 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031984a6-5359-4977-9655-4ba6185c524f None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:05.299541 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031984a6-5359-4977-9655-4ba6185c524f None admin] PATH_INFO: `/v3/domains/default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:05.300732 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031984a6-5359-4977-9655-4ba6185c524f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:05.302188 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031984a6-5359-4977-9655-4ba6185c524f None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:05.304051 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 103/194] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:44:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:05.304660 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031984a6-5359-4977-9655-4ba6185c524f None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:05.340278 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:05.373009 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a90c6-0e4c-4b6b-bb6c-b55eb9626d21 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:05.405645 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031984a6-5359-4977-9655-4ba6185c524f None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:05.406229 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1eb07-7ff8-4a4e-8d04-f533f9ac2667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:05.406888 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1eb07-7ff8-4a4e-8d04-f533f9ac2667 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:05.407158 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1eb07-7ff8-4a4e-8d04-f533f9ac2667 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:05.407421 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1eb07-7ff8-4a4e-8d04-f533f9ac2667 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:05.417235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e1eb07-7ff8-4a4e-8d04-f533f9ac2667 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:05.423024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-07e1eb07-7ff8-4a4e-8d04-f533f9ac2667 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:44:05.423024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:05.423024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:05.423024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:05.423024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:05.423024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:05.423024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:05.423024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:05.423024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:05.423024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:05.423024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:05.423024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:05.423024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:05.423024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:05.423024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:05.423024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:05.423915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:05.423915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:05.423915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:05.423915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:05.423915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:05.423915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:05.423915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:05.423915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:05.423915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:05.423915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:05.423915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:05.423915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:05.423915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:05.423915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:05.423915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:05.423915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:05.424688 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:05.424688 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:05.424688 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:05.424688 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:05.424688 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:05.424688 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:05.424688 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:05.424688 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:05.424688 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:05.424688 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:05.424688 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:44:05.424688 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:05.424688 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 92/195] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:05.426326 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1eb07-7ff8-4a4e-8d04-f533f9ac2667 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:05.441135 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92318af2-30dc-4706-8823-1930684b5067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:05.441741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92318af2-30dc-4706-8823-1930684b5067 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:05.442018 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92318af2-30dc-4706-8823-1930684b5067 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:05.442286 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92318af2-30dc-4706-8823-1930684b5067 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:05.467052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:05.509717 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92318af2-30dc-4706-8823-1930684b5067 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:05.511286 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92318af2-30dc-4706-8823-1930684b5067 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:05.527760 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1eb07-7ff8-4a4e-8d04-f533f9ac2667 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:05.545277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:05.561007 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75d19d-346d-4302-98a9-8eaca0f1eb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:05.561867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75d19d-346d-4302-98a9-8eaca0f1eb6c None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:05.562133 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75d19d-346d-4302-98a9-8eaca0f1eb6c None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:05.562414 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75d19d-346d-4302-98a9-8eaca0f1eb6c None admin] PATH_INFO: `/v3/users/9ede9378f81a45cc90c51b31386054c2` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:05.563558 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a75d19d-346d-4302-98a9-8eaca0f1eb6c None admin] RBAC: Authorizing `identity:get_user(user_id=9ede9378f81a45cc90c51b31386054c2)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:05.564983 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a75d19d-346d-4302-98a9-8eaca0f1eb6c None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:05.566578 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 93/196] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:44:05 2022] GET /identity/v3/users/9ede9378f81a45cc90c51b31386054c2 => generated 283 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:05.567308 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75d19d-346d-4302-98a9-8eaca0f1eb6c None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:05.614664 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 104/197] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:44:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:44:05.615514 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92318af2-30dc-4706-8823-1930684b5067 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:05.673595 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75d19d-346d-4302-98a9-8eaca0f1eb6c None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:05.681156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:05.708139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6edc2-ef0b-450c-8350-3047fb41b812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:05.708745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6edc2-ef0b-450c-8350-3047fb41b812 None admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:05.709018 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6edc2-ef0b-450c-8350-3047fb41b812 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:05.709286 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6edc2-ef0b-450c-8350-3047fb41b812 None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:05.709935 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6edc2-ef0b-450c-8350-3047fb41b812 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:05.711430 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6edc2-ef0b-450c-8350-3047fb41b812 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:05.715709 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92318af2-30dc-4706-8823-1930684b5067 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:05.749674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9039992-fcdb-49ef-b559-c3073f7d3736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:05.750212 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9039992-fcdb-49ef-b559-c3073f7d3736 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:05.750519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9039992-fcdb-49ef-b559-c3073f7d3736 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:05.750795 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9039992-fcdb-49ef-b559-c3073f7d3736 None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:05.751460 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9039992-fcdb-49ef-b559-c3073f7d3736 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:05.751761 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9039992-fcdb-49ef-b559-c3073f7d3736 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:05.753133 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9039992-fcdb-49ef-b559-c3073f7d3736 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:05.755258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:05.761725 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 105/198] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 31 10:44:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:05.762532 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9039992-fcdb-49ef-b559-c3073f7d3736 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:05.806740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 94/199] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 31 10:44:05 2022] POST /identity/v3/users => generated 297 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:44:05.807653 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6edc2-ef0b-450c-8350-3047fb41b812 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:05.879821 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9039992-fcdb-49ef-b559-c3073f7d3736 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:05.919530 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565e696-2ca9-45b6-a45f-4f58451504fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:05.920194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565e696-2ca9-45b6-a45f-4f58451504fc None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:05.920473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565e696-2ca9-45b6-a45f-4f58451504fc None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:05.920832 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565e696-2ca9-45b6-a45f-4f58451504fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:05.929298 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6edc2-ef0b-450c-8350-3047fb41b812 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:05.972472 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ed486-0c43-4868-9647-1a33d74e8d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:05.973110 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ed486-0c43-4868-9647-1a33d74e8d25 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:05.973386 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ed486-0c43-4868-9647-1a33d74e8d25 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:05.973652 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ed486-0c43-4868-9647-1a33d74e8d25 None admin] PATH_INFO: `/v3/projects/08b4540d439e44adb9398aec4130989a` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:05.974860 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ed486-0c43-4868-9647-1a33d74e8d25 None admin] RBAC: Authorizing `identity:get_project(project_id=08b4540d439e44adb9398aec4130989a)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:05.976316 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ed486-0c43-4868-9647-1a33d74e8d25 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:05.978139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 95/200] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:05 2022] GET /identity/v3/projects/08b4540d439e44adb9398aec4130989a => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:05.978876 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ed486-0c43-4868-9647-1a33d74e8d25 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:05.997900 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:06.009281 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565e696-2ca9-45b6-a45f-4f58451504fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:06.009281 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565e696-2ca9-45b6-a45f-4f58451504fc None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:06.011542 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:06.104334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ed486-0c43-4868-9647-1a33d74e8d25 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:06.137030 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 106/201] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:44:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:44:06.137987 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565e696-2ca9-45b6-a45f-4f58451504fc None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:06.155815 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a8d15-79d6-490c-9a80-71464ea10bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:06.156500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a8d15-79d6-490c-9a80-71464ea10bb7 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:06.156726 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a8d15-79d6-490c-9a80-71464ea10bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:06.156991 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a8d15-79d6-490c-9a80-71464ea10bb7 None admin] PATH_INFO: `/v3/users/3790cf1cdc3c4b95afd5eb64ded7322e` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:06.158238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9a8d15-79d6-490c-9a80-71464ea10bb7 None admin] RBAC: Authorizing `identity:get_user(user_id=3790cf1cdc3c4b95afd5eb64ded7322e)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:06.160165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9a8d15-79d6-490c-9a80-71464ea10bb7 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:06.161938 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 96/202] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:44:05 2022] GET /identity/v3/users/3790cf1cdc3c4b95afd5eb64ded7322e => generated 289 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:06.164046 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a8d15-79d6-490c-9a80-71464ea10bb7 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:06.278590 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:06.285106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565e696-2ca9-45b6-a45f-4f58451504fc None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:06.289872 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:06.294325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a8d15-79d6-490c-9a80-71464ea10bb7 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:06.332024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03b1fe-f768-4fe5-abdc-b2c3069a6c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:06.332920 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03b1fe-f768-4fe5-abdc-b2c3069a6c1a None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:06.333363 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03b1fe-f768-4fe5-abdc-b2c3069a6c1a None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:06.333885 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03b1fe-f768-4fe5-abdc-b2c3069a6c1a None admin] PATH_INFO: `/v3/projects/08b4540d439e44adb9398aec4130989a` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:06.335228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e03b1fe-f768-4fe5-abdc-b2c3069a6c1a None admin] RBAC: Authorizing `identity:get_project(project_id=08b4540d439e44adb9398aec4130989a)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:06.337089 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e03b1fe-f768-4fe5-abdc-b2c3069a6c1a None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:06.337729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebb511-225e-43bc-8da3-676d024040d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:06.338464 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebb511-225e-43bc-8da3-676d024040d0 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:06.338916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebb511-225e-43bc-8da3-676d024040d0 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:06.339291 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebb511-225e-43bc-8da3-676d024040d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:06.339585 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 97/203] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:06 2022] GET /identity/v3/projects/08b4540d439e44adb9398aec4130989a => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:06.340316 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ebb511-225e-43bc-8da3-676d024040d0 None admin] RBAC: Adding query filter params (role.id=13ca33d694c645d6920930d39dc52a0d, scope.project.id=08b4540d439e44adb9398aec4130989a, user.id=9ede9378f81a45cc90c51b31386054c2) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:06.340793 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ebb511-225e-43bc-8da3-676d024040d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:06.341021 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03b1fe-f768-4fe5-abdc-b2c3069a6c1a None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:06.343108 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ebb511-225e-43bc-8da3-676d024040d0 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:06.351939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 107/204] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:44:06 2022] GET /identity/v3/role_assignments?user.id=9ede9378f81a45cc90c51b31386054c2&scope.project.id=08b4540d439e44adb9398aec4130989a&role.id=13ca33d694c645d6920930d39dc52a0d => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:06.353118 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebb511-225e-43bc-8da3-676d024040d0 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:06.499260 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebb511-225e-43bc-8da3-676d024040d0 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:06.538961 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a779c-e340-4a7e-800d-9824095222f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:06.539594 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a779c-e340-4a7e-800d-9824095222f7 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:06.539867 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a779c-e340-4a7e-800d-9824095222f7 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:06.540142 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a779c-e340-4a7e-800d-9824095222f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:06.541340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a779c-e340-4a7e-800d-9824095222f7 None admin] RBAC: Adding query filter params (role.id=31d4b1dc082c41f7b6f18c338e7b51df, scope.project.id=08b4540d439e44adb9398aec4130989a, user.id=9ede9378f81a45cc90c51b31386054c2) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:06.541340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a779c-e340-4a7e-800d-9824095222f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:06.542620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a779c-e340-4a7e-800d-9824095222f7 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:06.547020 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03b1fe-f768-4fe5-abdc-b2c3069a6c1a None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:06.548631 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 108/205] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:44:06 2022] GET /identity/v3/role_assignments?user.id=9ede9378f81a45cc90c51b31386054c2&scope.project.id=08b4540d439e44adb9398aec4130989a&role.id=31d4b1dc082c41f7b6f18c338e7b51df => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:06.594607 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ba9a4-cae5-4153-938b-53f26c48aa95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:06.595238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ba9a4-cae5-4153-938b-53f26c48aa95 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:06.595582 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ba9a4-cae5-4153-938b-53f26c48aa95 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:06.595855 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ba9a4-cae5-4153-938b-53f26c48aa95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:06.635700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:06.678566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ba9a4-cae5-4153-938b-53f26c48aa95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:06.680116 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ba9a4-cae5-4153-938b-53f26c48aa95 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:06.796676 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 98/206] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:44:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:44:06.803564 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a779c-e340-4a7e-800d-9824095222f7 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:06.838829 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:06.933235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a779c-e340-4a7e-800d-9824095222f7 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:06.971988 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad6494-f7b1-4bb6-a0c7-87f08cca17e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:06.972683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad6494-f7b1-4bb6-a0c7-87f08cca17e2 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:06.972969 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad6494-f7b1-4bb6-a0c7-87f08cca17e2 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:06.973241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad6494-f7b1-4bb6-a0c7-87f08cca17e2 None admin] PATH_INFO: `/v3/projects/8d08421140694791b18a512ea16981a6` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:06.974385 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ad6494-f7b1-4bb6-a0c7-87f08cca17e2 None admin] RBAC: Authorizing `identity:get_project(project_id=8d08421140694791b18a512ea16981a6)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:06.975859 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ad6494-f7b1-4bb6-a0c7-87f08cca17e2 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:06.977959 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 109/207] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:06 2022] GET /identity/v3/projects/8d08421140694791b18a512ea16981a6 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:06.983981 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ba9a4-cae5-4153-938b-53f26c48aa95 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:07.052748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:07.089161 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ba9a4-cae5-4153-938b-53f26c48aa95 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:07.132015 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0868095-2b58-497a-8779-e30ddfee4fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:07.132636 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0868095-2b58-497a-8779-e30ddfee4fdc None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:07.132912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0868095-2b58-497a-8779-e30ddfee4fdc None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:07.133183 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0868095-2b58-497a-8779-e30ddfee4fdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:07.133915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0868095-2b58-497a-8779-e30ddfee4fdc None admin] RBAC: Adding query filter params (role.id=5585e95a315947d1ad314d7978b46e3a, scope.project.id=8d08421140694791b18a512ea16981a6, user.id=3790cf1cdc3c4b95afd5eb64ded7322e) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:07.134268 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0868095-2b58-497a-8779-e30ddfee4fdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:07.135796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0868095-2b58-497a-8779-e30ddfee4fdc None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:07.142330 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 99/208] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:44:06 2022] GET /identity/v3/role_assignments?user.id=3790cf1cdc3c4b95afd5eb64ded7322e&scope.project.id=8d08421140694791b18a512ea16981a6&role.id=5585e95a315947d1ad314d7978b46e3a => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:08.206516 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b572079-e843-4363-883c-2c548754d514 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:08.207474 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b572079-e843-4363-883c-2c548754d514 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:08.208017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b572079-e843-4363-883c-2c548754d514 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:08.209574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 110/209] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:08.215158 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3488b11-ea21-4368-92e0-2bc7adddc9e1 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:08.215998 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3488b11-ea21-4368-92e0-2bc7adddc9e1 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:08.216460 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3488b11-ea21-4368-92e0-2bc7adddc9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:08.235113 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-c3488b11-ea21-4368-92e0-2bc7adddc9e1 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:08.236644 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-c3488b11-ea21-4368-92e0-2bc7adddc9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:08.310437 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 100/210] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:08.315550 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f3592-8392-486e-89a1-186f48ba35e8 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:08.316078 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f3592-8392-486e-89a1-186f48ba35e8 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:08.316554 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f3592-8392-486e-89a1-186f48ba35e8 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:08.318145 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 111/211] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:08.323503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de8c22-443f-492c-b648-0343045cad6d None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:08.323790 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de8c22-443f-492c-b648-0343045cad6d None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:08.324063 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de8c22-443f-492c-b648-0343045cad6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:08.343330 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-23de8c22-443f-492c-b648-0343045cad6d None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:08.344526 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-23de8c22-443f-492c-b648-0343045cad6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:08.402006 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:08.413618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 101/212] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:08.597028 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f3592-8392-486e-89a1-186f48ba35e8 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:08.623500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bb4f6-81bd-4392-825d-ca7979b575e1 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:08.623948 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bb4f6-81bd-4392-825d-ca7979b575e1 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:08.624189 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bb4f6-81bd-4392-825d-ca7979b575e1 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:08.625893 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 102/213] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:44:08.631206 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1cf17c-bb3c-4f1f-a954-44b9bd66ef04 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:08.631515 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1cf17c-bb3c-4f1f-a954-44b9bd66ef04 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:08.631770 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1cf17c-bb3c-4f1f-a954-44b9bd66ef04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:08.651435 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-5d1cf17c-bb3c-4f1f-a954-44b9bd66ef04 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:08.652824 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-5d1cf17c-bb3c-4f1f-a954-44b9bd66ef04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:08.711130 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:08.718794 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f3592-8392-486e-89a1-186f48ba35e8 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:08.725365 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 103/214] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:44:08.727331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1cf17c-bb3c-4f1f-a954-44b9bd66ef04 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:08.754112 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ade1e-f2b2-41c8-8801-b1ad3855845d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:08.754775 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ade1e-f2b2-41c8-8801-b1ad3855845d None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:08.755050 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ade1e-f2b2-41c8-8801-b1ad3855845d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:08.755530 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ade1e-f2b2-41c8-8801-b1ad3855845d None admin] PATH_INFO: `/v3/domains/default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:08.756478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ade1e-f2b2-41c8-8801-b1ad3855845d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:08.757877 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ade1e-f2b2-41c8-8801-b1ad3855845d None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:08.763810 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 112/215] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:44:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:08.763810 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47753193-782f-4557-b36a-59a81d32ed84 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:08.763810 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47753193-782f-4557-b36a-59a81d32ed84 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:08.763810 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47753193-782f-4557-b36a-59a81d32ed84 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:08.763810 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 113/216] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:08.765335 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47753193-782f-4557-b36a-59a81d32ed84 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:08.837327 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1cf17c-bb3c-4f1f-a954-44b9bd66ef04 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:08.859221 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:08.873288 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47753193-782f-4557-b36a-59a81d32ed84 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:08.874127 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de56b9ab-9655-4fcd-8aab-d81add18b6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:08.874954 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56b9ab-9655-4fcd-8aab-d81add18b6c4 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:08.875236 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56b9ab-9655-4fcd-8aab-d81add18b6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:08.875528 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56b9ab-9655-4fcd-8aab-d81add18b6c4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:08.886453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de56b9ab-9655-4fcd-8aab-d81add18b6c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:08.893238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-de56b9ab-9655-4fcd-8aab-d81add18b6c4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:44:08.893238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:08.893238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:08.893238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:08.893238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:08.893238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:08.893238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:08.893238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:08.893238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:08.893238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:08.893238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:08.893238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:08.893238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:08.893238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:08.893238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:08.893238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:08.894145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:08.894145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:08.894145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:08.894145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:08.894145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:08.894145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:08.894145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:08.894145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:08.894145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:08.894145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:08.894145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:08.894145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:08.894145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:08.894145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:08.894145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:08.894145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:08.895617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:08.895617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:08.895617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:08.895617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:08.895617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:08.895617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:08.895617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:08.895617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:08.895617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:08.895617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:08.895617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:44:08.895617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:08.895617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 104/217] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:08.898142 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a5f38-6e57-4b62-9766-1a99dc25c6d4 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:08.898666 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a5f38-6e57-4b62-9766-1a99dc25c6d4 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:08.899140 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a5f38-6e57-4b62-9766-1a99dc25c6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:08.917249 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-f73a5f38-6e57-4b62-9766-1a99dc25c6d4 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:08.917438 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb073600-b8c2-497c-be09-a0b233532372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:08.917945 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb073600-b8c2-497c-be09-a0b233532372 None admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:08.918220 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb073600-b8c2-497c-be09-a0b233532372 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:08.918519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb073600-b8c2-497c-be09-a0b233532372 None admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:08.918628 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-f73a5f38-6e57-4b62-9766-1a99dc25c6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:08.919174 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb073600-b8c2-497c-be09-a0b233532372 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:08.920748 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb073600-b8c2-497c-be09-a0b233532372 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:08.984386 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 105/218] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:08.985014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a5f38-6e57-4b62-9766-1a99dc25c6d4 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:09.029948 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 114/219] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 31 10:44:08 2022] POST /identity/v3/users => generated 297 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 10:44:09.030974 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb073600-b8c2-497c-be09-a0b233532372 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:09.061558 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:09.114092 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a5f38-6e57-4b62-9766-1a99dc25c6d4 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:09.129192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:09.157482 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da15a7-1c4a-4599-8caf-d7c09b6264d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:09.158259 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da15a7-1c4a-4599-8caf-d7c09b6264d5 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:09.158693 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da15a7-1c4a-4599-8caf-d7c09b6264d5 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:09.159165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da15a7-1c4a-4599-8caf-d7c09b6264d5 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:09.160371 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da15a7-1c4a-4599-8caf-d7c09b6264d5 None admin] RBAC: Adding query filter params (name=member) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:09.160601 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da15a7-1c4a-4599-8caf-d7c09b6264d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:09.162111 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da15a7-1c4a-4599-8caf-d7c09b6264d5 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:09.178980 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 106/220] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 31 10:44:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:09.181165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da15a7-1c4a-4599-8caf-d7c09b6264d5 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:09.185544 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb073600-b8c2-497c-be09-a0b233532372 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:09.231093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3eaa0e-4e45-472b-bc78-0e79318713e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:09.231736 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3eaa0e-4e45-472b-bc78-0e79318713e3 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:09.231992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3eaa0e-4e45-472b-bc78-0e79318713e3 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:09.232264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3eaa0e-4e45-472b-bc78-0e79318713e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:09.249387 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3eaa0e-4e45-472b-bc78-0e79318713e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:09.256864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-da3eaa0e-4e45-472b-bc78-0e79318713e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:44:09.256864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:09.256864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:09.256864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:09.256864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:09.256864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:09.256864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:09.256864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:09.256864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:09.256864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:09.256864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:09.256864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:09.256864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:09.256864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:09.256864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:09.256864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:09.257710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:09.257710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:09.257710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:09.257710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:09.257710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:09.257710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:09.257710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:09.257710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:09.257710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:09.257710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:09.257710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:09.257710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:09.257710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:09.257710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:09.257710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:09.257710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:09.258573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:09.258573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:09.258573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:09.258573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:09.258573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:09.258573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:09.258573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:09.258573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:09.258573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:09.258573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:09.258573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:44:09.258573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:09.258573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 115/221] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:44:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:09.265417 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3eaa0e-4e45-472b-bc78-0e79318713e3 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:09.320310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da15a7-1c4a-4599-8caf-d7c09b6264d5 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:09.360052 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:09.364202 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bbb2f2-7044-4ded-b6d1-0a7dac32c3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:09.364844 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bbb2f2-7044-4ded-b6d1-0a7dac32c3d6 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:09.365115 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bbb2f2-7044-4ded-b6d1-0a7dac32c3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:09.365382 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bbb2f2-7044-4ded-b6d1-0a7dac32c3d6 None admin] PATH_INFO: `/v3/users/d7759a1e2e334613860406db8bb5efd9` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:09.383481 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bbb2f2-7044-4ded-b6d1-0a7dac32c3d6 None admin] RBAC: Authorizing `identity:get_user(user_id=d7759a1e2e334613860406db8bb5efd9)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:09.383690 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3eaa0e-4e45-472b-bc78-0e79318713e3 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:09.384952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bbb2f2-7044-4ded-b6d1-0a7dac32c3d6 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:09.386670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 107/222] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:44:09 2022] GET /identity/v3/users/d7759a1e2e334613860406db8bb5efd9 => generated 297 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:44:09.387436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bbb2f2-7044-4ded-b6d1-0a7dac32c3d6 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:09.426069 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac87589-0179-4f80-ae3b-ad0eba0ad340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:09.426761 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac87589-0179-4f80-ae3b-ad0eba0ad340 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:09.427042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac87589-0179-4f80-ae3b-ad0eba0ad340 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:09.427310 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac87589-0179-4f80-ae3b-ad0eba0ad340 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:09.428803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:09.443392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac87589-0179-4f80-ae3b-ad0eba0ad340 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:09.449499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-fac87589-0179-4f80-ae3b-ad0eba0ad340 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:44:09.449499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:09.449499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:09.449499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:09.449499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:09.449499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:09.449499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:09.449499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:09.449499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:09.449499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:09.449499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:09.449499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:09.449499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:09.449499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:09.449499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:09.449499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:09.450379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:09.450379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:09.450379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:09.450379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:09.450379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:09.450379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:09.450379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:09.450379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:09.450379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:09.450379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:09.450379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:09.450379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:09.450379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:09.450379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:09.450379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:09.450379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:09.452114 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:09.452114 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:09.452114 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:09.452114 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:09.452114 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:09.452114 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:09.452114 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:09.452114 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:09.452114 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:09.452114 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:09.452114 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:44:09.452114 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:09.452114 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 116/223] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 31 10:44:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:09.452914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac87589-0179-4f80-ae3b-ad0eba0ad340 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:09.534474 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bbb2f2-7044-4ded-b6d1-0a7dac32c3d6 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:09.572207 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac87589-0179-4f80-ae3b-ad0eba0ad340 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:09.581992 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9d053-fdc2-48cf-9f2a-d30cd12c028b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:09.582694 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9d053-fdc2-48cf-9f2a-d30cd12c028b None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:09.583438 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9d053-fdc2-48cf-9f2a-d30cd12c028b None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:09.583721 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9d053-fdc2-48cf-9f2a-d30cd12c028b None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:09.584789 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9d053-fdc2-48cf-9f2a-d30cd12c028b None admin] RBAC: Adding query filter params (name=admin) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:09.585102 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9d053-fdc2-48cf-9f2a-d30cd12c028b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:09.586856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9d053-fdc2-48cf-9f2a-d30cd12c028b None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:09.599614 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 108/224] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:09.600568 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9d053-fdc2-48cf-9f2a-d30cd12c028b None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:09.610819 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0017f33-396a-440e-b92e-96055daf9ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:09.611571 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0017f33-396a-440e-b92e-96055daf9ac9 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:09.611938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0017f33-396a-440e-b92e-96055daf9ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:09.612356 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0017f33-396a-440e-b92e-96055daf9ac9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:09.622254 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:09.688697 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0017f33-396a-440e-b92e-96055daf9ac9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:09.690413 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0017f33-396a-440e-b92e-96055daf9ac9 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:09.736322 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9d053-fdc2-48cf-9f2a-d30cd12c028b None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:09.738694 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:09.773112 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab8afd-30b7-4a70-a4a3-8155ae2f70b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:09.773720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab8afd-30b7-4a70-a4a3-8155ae2f70b3 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:09.773991 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab8afd-30b7-4a70-a4a3-8155ae2f70b3 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:09.774265 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab8afd-30b7-4a70-a4a3-8155ae2f70b3 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:09.774972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab8afd-30b7-4a70-a4a3-8155ae2f70b3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:09.775288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab8afd-30b7-4a70-a4a3-8155ae2f70b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:09.776647 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab8afd-30b7-4a70-a4a3-8155ae2f70b3 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:09.785684 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 109/225] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:44:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:09.786548 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab8afd-30b7-4a70-a4a3-8155ae2f70b3 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:09.803604 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 117/226] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:44:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:44:09.804883 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0017f33-396a-440e-b92e-96055daf9ac9 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:09.847372 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:09.895981 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab8afd-30b7-4a70-a4a3-8155ae2f70b3 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:09.911232 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0017f33-396a-440e-b92e-96055daf9ac9 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:09.932321 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2fdd6b-ac41-4f8e-980d-8309f74a698d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:09.933172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2fdd6b-ac41-4f8e-980d-8309f74a698d None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:09.933589 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2fdd6b-ac41-4f8e-980d-8309f74a698d None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:09.933993 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2fdd6b-ac41-4f8e-980d-8309f74a698d None admin] PATH_INFO: `/v3/users/9ede9378f81a45cc90c51b31386054c2` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:09.935491 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2fdd6b-ac41-4f8e-980d-8309f74a698d None admin] RBAC: Authorizing `identity:get_user(user_id=9ede9378f81a45cc90c51b31386054c2)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:09.937902 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2fdd6b-ac41-4f8e-980d-8309f74a698d None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:09.939850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 110/227] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:44:09 2022] GET /identity/v3/users/9ede9378f81a45cc90c51b31386054c2 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:09.940857 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2fdd6b-ac41-4f8e-980d-8309f74a698d None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:09.948429 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc345744-b89d-4340-b106-c6c30ef5cb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:09.949182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc345744-b89d-4340-b106-c6c30ef5cb67 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:09.949561 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc345744-b89d-4340-b106-c6c30ef5cb67 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:09.949930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc345744-b89d-4340-b106-c6c30ef5cb67 None admin] PATH_INFO: `/v3/users/9ede9378f81a45cc90c51b31386054c2` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:09.951222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc345744-b89d-4340-b106-c6c30ef5cb67 None admin] RBAC: Authorizing `identity:get_user(user_id=9ede9378f81a45cc90c51b31386054c2)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:09.953909 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc345744-b89d-4340-b106-c6c30ef5cb67 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:09.955497 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 118/228] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:44:09 2022] GET /identity/v3/users/9ede9378f81a45cc90c51b31386054c2 => generated 283 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:09.956335 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc345744-b89d-4340-b106-c6c30ef5cb67 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:09.970749 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:10.049877 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2fdd6b-ac41-4f8e-980d-8309f74a698d None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:10.060213 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc345744-b89d-4340-b106-c6c30ef5cb67 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:10.068170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:10.097356 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb9af2-4823-4ec0-a89e-7cbbc5ba17b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:10.098082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb9af2-4823-4ec0-a89e-7cbbc5ba17b4 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:10.098468 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb9af2-4823-4ec0-a89e-7cbbc5ba17b4 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:10.098819 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb9af2-4823-4ec0-a89e-7cbbc5ba17b4 None admin] PATH_INFO: `/v3/projects/08b4540d439e44adb9398aec4130989a` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:10.100125 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb9af2-4823-4ec0-a89e-7cbbc5ba17b4 None admin] RBAC: Authorizing `identity:get_project(project_id=08b4540d439e44adb9398aec4130989a)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:10.101666 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb9af2-4823-4ec0-a89e-7cbbc5ba17b4 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:10.103775 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 111/229] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:09 2022] GET /identity/v3/projects/08b4540d439e44adb9398aec4130989a => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:10.104684 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb9af2-4823-4ec0-a89e-7cbbc5ba17b4 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:10.116105 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4226278-1d0b-4d47-a25f-1559cb899371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:10.117934 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4226278-1d0b-4d47-a25f-1559cb899371 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:10.118348 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4226278-1d0b-4d47-a25f-1559cb899371 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:10.118800 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4226278-1d0b-4d47-a25f-1559cb899371 None admin] PATH_INFO: `/v3/projects/08b4540d439e44adb9398aec4130989a` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:10.120111 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4226278-1d0b-4d47-a25f-1559cb899371 None admin] RBAC: Authorizing `identity:get_project(project_id=08b4540d439e44adb9398aec4130989a)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:10.122365 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4226278-1d0b-4d47-a25f-1559cb899371 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:10.125843 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 119/230] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:09 2022] GET /identity/v3/projects/08b4540d439e44adb9398aec4130989a => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:10.126799 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4226278-1d0b-4d47-a25f-1559cb899371 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:10.214933 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:10.225858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb9af2-4823-4ec0-a89e-7cbbc5ba17b4 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:10.233608 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4226278-1d0b-4d47-a25f-1559cb899371 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:10.267046 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d002d28a-2d88-48db-8730-79894f76ef33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:10.267690 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d002d28a-2d88-48db-8730-79894f76ef33 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:10.268033 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d002d28a-2d88-48db-8730-79894f76ef33 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:10.268237 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d002d28a-2d88-48db-8730-79894f76ef33 None admin] PATH_INFO: `/v3/projects/08b4540d439e44adb9398aec4130989a` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:10.269508 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d002d28a-2d88-48db-8730-79894f76ef33 None admin] RBAC: Authorizing `identity:get_project(project_id=08b4540d439e44adb9398aec4130989a)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:10.271114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d002d28a-2d88-48db-8730-79894f76ef33 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:10.277516 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 112/231] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:10 2022] GET /identity/v3/projects/08b4540d439e44adb9398aec4130989a => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:10.277516 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d002d28a-2d88-48db-8730-79894f76ef33 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:10.284204 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9afb6-07c3-4df9-978e-4704892c3d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:10.285031 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9afb6-07c3-4df9-978e-4704892c3d2f None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:10.285473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9afb6-07c3-4df9-978e-4704892c3d2f None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:10.285878 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9afb6-07c3-4df9-978e-4704892c3d2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:10.286766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9afb6-07c3-4df9-978e-4704892c3d2f None admin] RBAC: Adding query filter params (role.id=2278da3c52644a819d210b5f9c20b6c1, scope.project.id=08b4540d439e44adb9398aec4130989a, user.id=d7759a1e2e334613860406db8bb5efd9) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:10.287210 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9afb6-07c3-4df9-978e-4704892c3d2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:10.288938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9afb6-07c3-4df9-978e-4704892c3d2f None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:10.295311 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 120/232] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:44:10 2022] GET /identity/v3/role_assignments?user.id=d7759a1e2e334613860406db8bb5efd9&scope.project.id=08b4540d439e44adb9398aec4130989a&role.id=2278da3c52644a819d210b5f9c20b6c1 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:10.296650 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9afb6-07c3-4df9-978e-4704892c3d2f None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:10.349782 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:10.400394 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d002d28a-2d88-48db-8730-79894f76ef33 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:10.439792 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9afb6-07c3-4df9-978e-4704892c3d2f None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:10.478780 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb17f45-13ca-4f19-9ae1-974aec0303ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:10.479936 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb17f45-13ca-4f19-9ae1-974aec0303ac None admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:10.480411 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb17f45-13ca-4f19-9ae1-974aec0303ac None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:10.480875 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb17f45-13ca-4f19-9ae1-974aec0303ac None admin] PATH_INFO: `/v3/projects/08b4540d439e44adb9398aec4130989a/users/9ede9378f81a45cc90c51b31386054c2/roles/31d4b1dc082c41f7b6f18c338e7b51df` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:10.483642 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb17f45-13ca-4f19-9ae1-974aec0303ac None admin] RBAC: Authorizing `identity:create_grant(project_id=08b4540d439e44adb9398aec4130989a, user_id=9ede9378f81a45cc90c51b31386054c2, role_id=31d4b1dc082c41f7b6f18c338e7b51df)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:10.485446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb17f45-13ca-4f19-9ae1-974aec0303ac None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:10.494813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885c632-31ec-4cf7-822b-e09c6231b399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:10.496251 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885c632-31ec-4cf7-822b-e09c6231b399 None admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:10.496785 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885c632-31ec-4cf7-822b-e09c6231b399 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:10.497292 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885c632-31ec-4cf7-822b-e09c6231b399 None admin] PATH_INFO: `/v3/projects/08b4540d439e44adb9398aec4130989a/users/9ede9378f81a45cc90c51b31386054c2/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:10.500445 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9885c632-31ec-4cf7-822b-e09c6231b399 None admin] RBAC: Authorizing `identity:create_grant(project_id=08b4540d439e44adb9398aec4130989a, user_id=9ede9378f81a45cc90c51b31386054c2, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:10.502331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9885c632-31ec-4cf7-822b-e09c6231b399 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:10.598294 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 121/233] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 31 10:44:10 2022] PUT /identity/v3/projects/08b4540d439e44adb9398aec4130989a/users/9ede9378f81a45cc90c51b31386054c2/roles/31d4b1dc082c41f7b6f18c338e7b51df => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:44:10.631854 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 113/234] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 31 10:44:10 2022] PUT /identity/v3/projects/08b4540d439e44adb9398aec4130989a/users/9ede9378f81a45cc90c51b31386054c2/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:44:11.325423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca949a4d-e6b7-4b6d-931e-e1ccbf52b894 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:11.325931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca949a4d-e6b7-4b6d-931e-e1ccbf52b894 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:11.326003 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca949a4d-e6b7-4b6d-931e-e1ccbf52b894 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:11.327647 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 122/235] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:11.333009 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3893829-0ff2-4fc3-bf52-de601bf5c714 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:11.333292 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3893829-0ff2-4fc3-bf52-de601bf5c714 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:11.333561 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3893829-0ff2-4fc3-bf52-de601bf5c714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:11.355202 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-c3893829-0ff2-4fc3-bf52-de601bf5c714 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:11.356490 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-c3893829-0ff2-4fc3-bf52-de601bf5c714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:11.374808 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:11.379183 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63685838-faa8-442f-aa16-db809b4f240b None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:11.379519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63685838-faa8-442f-aa16-db809b4f240b None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:11.379804 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63685838-faa8-442f-aa16-db809b4f240b None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:11.381424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 123/236] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:11.386678 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd4475-28f9-4f3d-b3d9-d4b28cef13e5 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:11.401528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd4475-28f9-4f3d-b3d9-d4b28cef13e5 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:11.401528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd4475-28f9-4f3d-b3d9-d4b28cef13e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:11.415224 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-88dd4475-28f9-4f3d-b3d9-d4b28cef13e5 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:11.416496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-88dd4475-28f9-4f3d-b3d9-d4b28cef13e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:11.465496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 114/237] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:11.507575 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 124/238] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:44:11.807022 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3893829-0ff2-4fc3-bf52-de601bf5c714 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:11.825704 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd4475-28f9-4f3d-b3d9-d4b28cef13e5 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:11.879910 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:11.996456 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3893829-0ff2-4fc3-bf52-de601bf5c714 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:12.016562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd4475-28f9-4f3d-b3d9-d4b28cef13e5 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:12.065126 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758feea-b622-466b-97a2-b241926a8b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:12.065683 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758feea-b622-466b-97a2-b241926a8b68 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:12.065958 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758feea-b622-466b-97a2-b241926a8b68 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:12.066251 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758feea-b622-466b-97a2-b241926a8b68 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:12.080630 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec4ee0-0a67-42df-83a6-0dedcb9682cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:12.081644 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec4ee0-0a67-42df-83a6-0dedcb9682cc None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:12.082123 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec4ee0-0a67-42df-83a6-0dedcb9682cc None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:12.082633 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec4ee0-0a67-42df-83a6-0dedcb9682cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:12.084694 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec4ee0-0a67-42df-83a6-0dedcb9682cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:12.086606 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec4ee0-0a67-42df-83a6-0dedcb9682cc None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:12.093471 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2758feea-b622-466b-97a2-b241926a8b68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:12.093782 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 125/239] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:44:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:44:12.095331 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec4ee0-0a67-42df-83a6-0dedcb9682cc None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:12.107413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-2758feea-b622-466b-97a2-b241926a8b68 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:44:12.107413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:12.107413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:12.107413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:12.107413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:12.107413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:12.107413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:12.107413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:12.107413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:12.107413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:12.107413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:12.107413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:12.107413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:12.107413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:12.107413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:12.107413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:12.108340 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:12.108340 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:12.108340 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:12.108340 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:12.108340 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:12.108340 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:12.108340 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:12.108340 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:12.108340 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:12.108340 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:12.108340 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:12.108340 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:12.108340 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:12.108340 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:12.108340 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:12.108340 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:12.109122 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:12.109122 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:12.109122 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:12.109122 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:12.109122 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:12.109122 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:12.109122 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:12.109122 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:12.109122 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:12.109122 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:12.109122 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:44:12.109122 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:12.109122 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 115/240] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:12.123921 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758feea-b622-466b-97a2-b241926a8b68 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:12.152489 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:12.234779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:12.243526 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec4ee0-0a67-42df-83a6-0dedcb9682cc None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:12.309962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830adfde-b868-4d00-878e-6eaffbec9c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:12.310720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830adfde-b868-4d00-878e-6eaffbec9c3a None admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:12.311103 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830adfde-b868-4d00-878e-6eaffbec9c3a None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:12.311381 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830adfde-b868-4d00-878e-6eaffbec9c3a None admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:12.312053 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830adfde-b868-4d00-878e-6eaffbec9c3a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:12.313502 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830adfde-b868-4d00-878e-6eaffbec9c3a None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:12.330781 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758feea-b622-466b-97a2-b241926a8b68 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:12.372299 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdab967-341a-42e6-aff1-f34e69b3ec97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:12.373306 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdab967-341a-42e6-aff1-f34e69b3ec97 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:12.373861 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdab967-341a-42e6-aff1-f34e69b3ec97 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:12.374359 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdab967-341a-42e6-aff1-f34e69b3ec97 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:12.375277 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdab967-341a-42e6-aff1-f34e69b3ec97 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:12.375860 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdab967-341a-42e6-aff1-f34e69b3ec97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:12.377772 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdab967-341a-42e6-aff1-f34e69b3ec97 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:12.387429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 116/241] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 31 10:44:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:12.393242 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdab967-341a-42e6-aff1-f34e69b3ec97 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:12.427037 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 126/242] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 31 10:44:12 2022] POST /identity/v3/users => generated 293 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:44:12.599449 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:12.648153 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d046c8-6124-454e-ba86-bf8ffec86ad2 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:12.648153 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d046c8-6124-454e-ba86-bf8ffec86ad2 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:12.648153 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d046c8-6124-454e-ba86-bf8ffec86ad2 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:12.648153 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 127/243] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:44:12.655177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2713d-6860-4aec-93f6-96c9bbdf19ce None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:12.655177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2713d-6860-4aec-93f6-96c9bbdf19ce None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:12.655177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2713d-6860-4aec-93f6-96c9bbdf19ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:12.676548 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-a2c2713d-6860-4aec-93f6-96c9bbdf19ce None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:12.679002 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a2c2713d-6860-4aec-93f6-96c9bbdf19ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:12.692347 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdab967-341a-42e6-aff1-f34e69b3ec97 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:12.745947 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830456e-b860-4803-a409-f3c1b589c00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:12.747584 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830456e-b860-4803-a409-f3c1b589c00f None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:12.748443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830456e-b860-4803-a409-f3c1b589c00f None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:12.750036 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830456e-b860-4803-a409-f3c1b589c00f None admin] PATH_INFO: `/v3/users/be08a1a635ea4edf8ac4f73b7eefa049` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:12.771700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a830456e-b860-4803-a409-f3c1b589c00f None admin] RBAC: Authorizing `identity:get_user(user_id=be08a1a635ea4edf8ac4f73b7eefa049)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:12.776144 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a830456e-b860-4803-a409-f3c1b589c00f None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:12.783366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 117/244] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:44:12 2022] GET /identity/v3/users/be08a1a635ea4edf8ac4f73b7eefa049 => generated 297 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:12.786623 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830456e-b860-4803-a409-f3c1b589c00f None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:12.819096 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 128/245] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:44:13.017158 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830456e-b860-4803-a409-f3c1b589c00f None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:13.067168 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:13.077871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212ff0a-1ed8-4010-b749-40327600320e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:13.078833 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212ff0a-1ed8-4010-b749-40327600320e None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:13.084654 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212ff0a-1ed8-4010-b749-40327600320e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:13.084654 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212ff0a-1ed8-4010-b749-40327600320e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:13.146548 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c2713d-6860-4aec-93f6-96c9bbdf19ce None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:13.165294 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7212ff0a-1ed8-4010-b749-40327600320e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:13.166852 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7212ff0a-1ed8-4010-b749-40327600320e None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:13.258112 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:13.284149 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c2713d-6860-4aec-93f6-96c9bbdf19ce None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:13.304974 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 118/246] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:44:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:44:13.306349 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cbc96-7d66-4909-b62f-af98e8a294ad None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:13.306660 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cbc96-7d66-4909-b62f-af98e8a294ad None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:13.306932 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cbc96-7d66-4909-b62f-af98e8a294ad None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:13.308228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 119/247] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:13.309380 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630c780-2e45-4035-8139-f5735a0e34c9 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:13.309697 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630c780-2e45-4035-8139-f5735a0e34c9 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:13.309976 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630c780-2e45-4035-8139-f5735a0e34c9 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:13.312749 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 120/248] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:13.314339 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73becde8-ecd6-4f8a-bec9-fcd3d8c13f5a None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:13.314655 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73becde8-ecd6-4f8a-bec9-fcd3d8c13f5a None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:13.314929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73becde8-ecd6-4f8a-bec9-fcd3d8c13f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:13.326243 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c59170-3776-4540-b8cb-2f06e76e2b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:13.327650 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c59170-3776-4540-b8cb-2f06e76e2b67 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:13.328020 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c59170-3776-4540-b8cb-2f06e76e2b67 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:13.328284 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c59170-3776-4540-b8cb-2f06e76e2b67 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:13.337387 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-73becde8-ecd6-4f8a-bec9-fcd3d8c13f5a None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:13.338668 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-73becde8-ecd6-4f8a-bec9-fcd3d8c13f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:13.340077 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c59170-3776-4540-b8cb-2f06e76e2b67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:13.346482 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-b0c59170-3776-4540-b8cb-2f06e76e2b67 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:44:13.346482 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:13.346482 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:13.346482 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:13.346482 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:13.346482 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:13.346482 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:13.346482 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:13.346482 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:13.346482 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:13.346482 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:13.346482 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:13.346482 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:13.346482 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:13.346482 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:13.346482 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:13.348932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:13.348932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:13.348932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:13.348932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:13.348932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:13.348932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:13.348932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:13.348932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:13.348932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:13.348932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:13.348932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:13.348932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:13.348932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:13.348932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:13.348932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:13.348932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:13.349856 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:13.349856 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:13.349856 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:13.349856 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:13.349856 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:13.349856 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:13.349856 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:13.349856 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:13.349856 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:13.349856 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:13.349856 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:44:13.349856 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:13.350765 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 129/249] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:13.351518 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c59170-3776-4540-b8cb-2f06e76e2b67 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:13.377062 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:13.428654 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 121/250] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:13.430194 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036357c-9d99-4d0b-9dd4-5a4c035aa23a None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:13.430680 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036357c-9d99-4d0b-9dd4-5a4c035aa23a None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:13.431123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036357c-9d99-4d0b-9dd4-5a4c035aa23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:13.475454 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-4036357c-9d99-4d0b-9dd4-5a4c035aa23a None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:13.476676 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-4036357c-9d99-4d0b-9dd4-5a4c035aa23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:13.491513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c59170-3776-4540-b8cb-2f06e76e2b67 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:13.541171 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359987c-e135-4e5f-959a-e29416a79881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:13.541734 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359987c-e135-4e5f-959a-e29416a79881 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:13.542054 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359987c-e135-4e5f-959a-e29416a79881 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:13.542331 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359987c-e135-4e5f-959a-e29416a79881 None admin] PATH_INFO: `/v3/projects/08b4540d439e44adb9398aec4130989a` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:13.544053 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7359987c-e135-4e5f-959a-e29416a79881 None admin] RBAC: Authorizing `identity:get_project(project_id=08b4540d439e44adb9398aec4130989a)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:13.546766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7359987c-e135-4e5f-959a-e29416a79881 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:13.550468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 130/251] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:13 2022] GET /identity/v3/projects/08b4540d439e44adb9398aec4130989a => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:13.552137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359987c-e135-4e5f-959a-e29416a79881 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:13.578424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:13.684452 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359987c-e135-4e5f-959a-e29416a79881 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:13.708104 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 122/252] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:13.709303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036357c-9d99-4d0b-9dd4-5a4c035aa23a None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:13.724460 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a82eb9-dafb-44dd-8965-5f270e0c4941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:13.725079 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a82eb9-dafb-44dd-8965-5f270e0c4941 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:13.725359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a82eb9-dafb-44dd-8965-5f270e0c4941 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:13.725637 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a82eb9-dafb-44dd-8965-5f270e0c4941 None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:13.726774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a82eb9-dafb-44dd-8965-5f270e0c4941 None admin] RBAC: Adding query filter params (name=member) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:13.727143 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a82eb9-dafb-44dd-8965-5f270e0c4941 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:13.728641 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a82eb9-dafb-44dd-8965-5f270e0c4941 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:13.739288 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 131/253] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 31 10:44:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:13.739965 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a82eb9-dafb-44dd-8965-5f270e0c4941 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:13.823339 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:13.869079 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a82eb9-dafb-44dd-8965-5f270e0c4941 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:13.938567 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:13.938920 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d44531-1d73-4d2e-accb-b98e81cc764c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:13.939783 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d44531-1d73-4d2e-accb-b98e81cc764c None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:13.939890 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d44531-1d73-4d2e-accb-b98e81cc764c None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:13.940153 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d44531-1d73-4d2e-accb-b98e81cc764c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:13.957184 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036357c-9d99-4d0b-9dd4-5a4c035aa23a None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:13.969655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d44531-1d73-4d2e-accb-b98e81cc764c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:13.975243 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-e5d44531-1d73-4d2e-accb-b98e81cc764c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:44:13.975243 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:13.975243 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:13.975243 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:13.975243 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:13.975243 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:13.975243 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:13.975243 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:13.975243 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:13.975243 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:13.975243 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:13.975243 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:13.975243 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:13.975243 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:13.975243 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:13.975243 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:13.976096 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:13.976096 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:13.976096 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:13.976096 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:13.976096 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:13.976096 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:13.976096 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:13.976096 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:13.976096 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:13.976096 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:13.976096 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:13.976096 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:13.976096 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:13.976096 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:13.976096 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:13.976096 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:13.976869 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:13.976869 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:13.976869 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:13.976869 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:13.976869 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:13.976869 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:13.976869 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:13.976869 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:13.976869 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:13.976869 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:13.976869 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:44:13.976869 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:13.976869 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 132/254] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 31 10:44:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:13.980841 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d44531-1d73-4d2e-accb-b98e81cc764c None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:14.078523 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe94b4a-4f1f-4514-876c-d8a264cfcbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:14.079057 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe94b4a-4f1f-4514-876c-d8a264cfcbee None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:14.079331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe94b4a-4f1f-4514-876c-d8a264cfcbee None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:14.079599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe94b4a-4f1f-4514-876c-d8a264cfcbee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:14.080325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe94b4a-4f1f-4514-876c-d8a264cfcbee None admin] RBAC: Adding query filter params (role.id=5585e95a315947d1ad314d7978b46e3a, scope.project.id=08b4540d439e44adb9398aec4130989a, user.id=be08a1a635ea4edf8ac4f73b7eefa049) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:14.080627 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe94b4a-4f1f-4514-876c-d8a264cfcbee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:14.082080 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe94b4a-4f1f-4514-876c-d8a264cfcbee None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:14.088105 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 123/255] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:44:13 2022] GET /identity/v3/role_assignments?user.id=be08a1a635ea4edf8ac4f73b7eefa049&scope.project.id=08b4540d439e44adb9398aec4130989a&role.id=5585e95a315947d1ad314d7978b46e3a => generated 259 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:14.088874 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe94b4a-4f1f-4514-876c-d8a264cfcbee None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:14.101358 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d44531-1d73-4d2e-accb-b98e81cc764c None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:14.162819 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:14.172969 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af933468-27e0-4933-bbc1-62305de4bd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:14.173668 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af933468-27e0-4933-bbc1-62305de4bd94 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:14.173951 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af933468-27e0-4933-bbc1-62305de4bd94 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:14.174226 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af933468-27e0-4933-bbc1-62305de4bd94 None admin] PATH_INFO: `/v3/users/d7759a1e2e334613860406db8bb5efd9` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:14.176919 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af933468-27e0-4933-bbc1-62305de4bd94 None admin] RBAC: Authorizing `identity:get_user(user_id=d7759a1e2e334613860406db8bb5efd9)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:14.178595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af933468-27e0-4933-bbc1-62305de4bd94 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:14.180282 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 133/256] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:44:13 2022] GET /identity/v3/users/d7759a1e2e334613860406db8bb5efd9 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:14.181073 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af933468-27e0-4933-bbc1-62305de4bd94 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:14.295543 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe94b4a-4f1f-4514-876c-d8a264cfcbee None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:14.334250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9391f5a-1f31-4803-92bc-69ae9cc8c6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:14.334806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9391f5a-1f31-4803-92bc-69ae9cc8c6df None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:14.335087 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9391f5a-1f31-4803-92bc-69ae9cc8c6df None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:14.335514 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9391f5a-1f31-4803-92bc-69ae9cc8c6df None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:14.336696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9391f5a-1f31-4803-92bc-69ae9cc8c6df None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:14.337011 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9391f5a-1f31-4803-92bc-69ae9cc8c6df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:14.338663 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9391f5a-1f31-4803-92bc-69ae9cc8c6df None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:14.349305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 124/257] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:44:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:14.350272 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9391f5a-1f31-4803-92bc-69ae9cc8c6df None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:14.382817 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:14.411249 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af933468-27e0-4933-bbc1-62305de4bd94 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:14.489656 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ea04c-420d-4787-bcbb-57f5f7738c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:14.490359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ea04c-420d-4787-bcbb-57f5f7738c21 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:14.490682 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ea04c-420d-4787-bcbb-57f5f7738c21 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:14.491052 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ea04c-420d-4787-bcbb-57f5f7738c21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:14.503019 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077ea04c-420d-4787-bcbb-57f5f7738c21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:14.508522 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-077ea04c-420d-4787-bcbb-57f5f7738c21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:44:14.508522 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:14.508522 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:14.508522 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:14.508522 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:14.508522 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:14.508522 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:14.508522 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:14.508522 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:14.508522 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:14.508522 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:14.508522 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:14.508522 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:14.508522 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:14.508522 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:14.508522 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:14.509392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:14.509392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:14.509392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:14.509392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:14.509392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:14.509392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:14.509392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:14.509392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:14.509392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:14.509392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:14.509392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:14.509392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:14.509392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:14.509392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:14.509392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:14.509392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:14.510622 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:14.510622 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:14.510622 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:14.510622 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:14.510622 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:14.510622 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:14.510622 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:14.510622 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:14.510622 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:14.510622 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:14.510622 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:44:14.510622 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:14.511296 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9391f5a-1f31-4803-92bc-69ae9cc8c6df None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:14.511384 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 134/258] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:44:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:14.511384 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ea04c-420d-4787-bcbb-57f5f7738c21 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:14.571809 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3ed2b-a490-48f6-a0e9-9796d53da0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:14.572452 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3ed2b-a490-48f6-a0e9-9796d53da0bb None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:14.572735 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3ed2b-a490-48f6-a0e9-9796d53da0bb None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:14.573018 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3ed2b-a490-48f6-a0e9-9796d53da0bb None admin] PATH_INFO: `/v3/projects/08b4540d439e44adb9398aec4130989a` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:14.574181 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd3ed2b-a490-48f6-a0e9-9796d53da0bb None admin] RBAC: Authorizing `identity:get_project(project_id=08b4540d439e44adb9398aec4130989a)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:14.575814 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd3ed2b-a490-48f6-a0e9-9796d53da0bb None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:14.577681 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 125/259] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:14 2022] GET /identity/v3/projects/08b4540d439e44adb9398aec4130989a => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:14.578529 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3ed2b-a490-48f6-a0e9-9796d53da0bb None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:14.610844 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:14.658779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ea04c-420d-4787-bcbb-57f5f7738c21 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:14.697277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c50d73-3d1e-4459-baf6-d0dd3eee04f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:14.697915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c50d73-3d1e-4459-baf6-d0dd3eee04f8 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:14.698194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c50d73-3d1e-4459-baf6-d0dd3eee04f8 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:14.698515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c50d73-3d1e-4459-baf6-d0dd3eee04f8 None admin] PATH_INFO: `/v3/users/9ede9378f81a45cc90c51b31386054c2` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:14.699673 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c50d73-3d1e-4459-baf6-d0dd3eee04f8 None admin] RBAC: Authorizing `identity:get_user(user_id=9ede9378f81a45cc90c51b31386054c2)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:14.701211 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c50d73-3d1e-4459-baf6-d0dd3eee04f8 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:14.703204 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 135/260] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:44:14 2022] GET /identity/v3/users/9ede9378f81a45cc90c51b31386054c2 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:14.703971 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c50d73-3d1e-4459-baf6-d0dd3eee04f8 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:14.750800 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:14.756611 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3ed2b-a490-48f6-a0e9-9796d53da0bb None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:14.827241 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25dcc92-6fd7-4916-a601-ddb2ad851fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:14.827823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25dcc92-6fd7-4916-a601-ddb2ad851fb8 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:14.828103 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25dcc92-6fd7-4916-a601-ddb2ad851fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:14.828377 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25dcc92-6fd7-4916-a601-ddb2ad851fb8 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:14.829051 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25dcc92-6fd7-4916-a601-ddb2ad851fb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:14.829352 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25dcc92-6fd7-4916-a601-ddb2ad851fb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:14.830762 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25dcc92-6fd7-4916-a601-ddb2ad851fb8 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:14.856277 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 126/261] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:14.857180 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25dcc92-6fd7-4916-a601-ddb2ad851fb8 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:14.927647 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c50d73-3d1e-4459-baf6-d0dd3eee04f8 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:14.985677 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25dcc92-6fd7-4916-a601-ddb2ad851fb8 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:14.987490 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:15.007603 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e569d21-0686-4c50-ab32-0bdfd4118367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:15.008285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e569d21-0686-4c50-ab32-0bdfd4118367 None admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:15.008569 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e569d21-0686-4c50-ab32-0bdfd4118367 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:15.008849 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e569d21-0686-4c50-ab32-0bdfd4118367 None admin] PATH_INFO: `/v3/projects/08b4540d439e44adb9398aec4130989a/users/d7759a1e2e334613860406db8bb5efd9/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:15.014560 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e569d21-0686-4c50-ab32-0bdfd4118367 None admin] RBAC: Authorizing `identity:create_grant(project_id=08b4540d439e44adb9398aec4130989a, user_id=d7759a1e2e334613860406db8bb5efd9, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:15.016351 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e569d21-0686-4c50-ab32-0bdfd4118367 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:15.026063 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0469b53-1967-4643-8de0-78b4d967b626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:15.026729 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0469b53-1967-4643-8de0-78b4d967b626 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:15.027090 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0469b53-1967-4643-8de0-78b4d967b626 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:15.027780 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0469b53-1967-4643-8de0-78b4d967b626 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:15.044667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:15.102757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0469b53-1967-4643-8de0-78b4d967b626 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:15.104239 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0469b53-1967-4643-8de0-78b4d967b626 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:15.118791 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 136/262] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 31 10:44:14 2022] PUT /identity/v3/projects/08b4540d439e44adb9398aec4130989a/users/d7759a1e2e334613860406db8bb5efd9/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:44:15.119693 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e569d21-0686-4c50-ab32-0bdfd4118367 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:15.231954 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 127/263] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:44:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:44:15.233785 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1063907-8cbf-47de-a359-eb48337a9e73 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:15.234072 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1063907-8cbf-47de-a359-eb48337a9e73 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:15.234342 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1063907-8cbf-47de-a359-eb48337a9e73 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:15.241978 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 128/264] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:15.242649 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1063907-8cbf-47de-a359-eb48337a9e73 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:15.332555 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:15.360701 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e569d21-0686-4c50-ab32-0bdfd4118367 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:15.397951 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1063907-8cbf-47de-a359-eb48337a9e73 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:15.413419 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5455f5c-ef1a-41f1-80c7-c937c00f5937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:15.414071 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5455f5c-ef1a-41f1-80c7-c937c00f5937 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:15.414353 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5455f5c-ef1a-41f1-80c7-c937c00f5937 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:15.414653 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5455f5c-ef1a-41f1-80c7-c937c00f5937 None admin] PATH_INFO: `/v3/users/9ede9378f81a45cc90c51b31386054c2` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:15.417980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5455f5c-ef1a-41f1-80c7-c937c00f5937 None admin] RBAC: Authorizing `identity:get_user(user_id=9ede9378f81a45cc90c51b31386054c2)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:15.417980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5455f5c-ef1a-41f1-80c7-c937c00f5937 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:15.421105 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 137/265] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:44:15 2022] GET /identity/v3/users/9ede9378f81a45cc90c51b31386054c2 => generated 283 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:15.422133 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd126a-094f-48d2-a1d4-249e29f6f12e None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:15.422404 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd126a-094f-48d2-a1d4-249e29f6f12e None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:15.422680 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd126a-094f-48d2-a1d4-249e29f6f12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:15.458158 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-bcfd126a-094f-48d2-a1d4-249e29f6f12e None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:15.459408 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-bcfd126a-094f-48d2-a1d4-249e29f6f12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:15.464275 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac387eea-ba5d-4c53-899a-abd0416830b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:15.464918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac387eea-ba5d-4c53-899a-abd0416830b9 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:15.465193 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac387eea-ba5d-4c53-899a-abd0416830b9 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:15.465467 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac387eea-ba5d-4c53-899a-abd0416830b9 None admin] PATH_INFO: `/v3/projects/08b4540d439e44adb9398aec4130989a` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:15.467187 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac387eea-ba5d-4c53-899a-abd0416830b9 None admin] RBAC: Authorizing `identity:get_project(project_id=08b4540d439e44adb9398aec4130989a)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:15.469144 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac387eea-ba5d-4c53-899a-abd0416830b9 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:15.474997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 129/266] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:15 2022] GET /identity/v3/projects/08b4540d439e44adb9398aec4130989a => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:15.475818 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac387eea-ba5d-4c53-899a-abd0416830b9 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:15.533159 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:15.552602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 138/267] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:15.553920 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a9b50-c575-48d4-a110-9353864605c8 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:15.554207 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a9b50-c575-48d4-a110-9353864605c8 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:15.554511 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a9b50-c575-48d4-a110-9353864605c8 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:15.555892 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 139/268] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:15.556442 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a9b50-c575-48d4-a110-9353864605c8 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:15.617145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac387eea-ba5d-4c53-899a-abd0416830b9 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:15.633370 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:15.667664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729edd5-bf82-4e3a-9c0a-b4f8d2061b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:15.668291 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729edd5-bf82-4e3a-9c0a-b4f8d2061b24 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:15.668575 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729edd5-bf82-4e3a-9c0a-b4f8d2061b24 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:15.668853 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729edd5-bf82-4e3a-9c0a-b4f8d2061b24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:15.698928 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a9b50-c575-48d4-a110-9353864605c8 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:15.766097 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddefde1c-ad81-4e9c-95aa-7752a3cd7d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:15.766742 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddefde1c-ad81-4e9c-95aa-7752a3cd7d56 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:15.767025 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddefde1c-ad81-4e9c-95aa-7752a3cd7d56 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:15.767300 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddefde1c-ad81-4e9c-95aa-7752a3cd7d56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:15.768036 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddefde1c-ad81-4e9c-95aa-7752a3cd7d56 None admin] RBAC: Adding query filter params (role.id=31d4b1dc082c41f7b6f18c338e7b51df, scope.project.id=08b4540d439e44adb9398aec4130989a, user.id=9ede9378f81a45cc90c51b31386054c2) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:15.768346 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddefde1c-ad81-4e9c-95aa-7752a3cd7d56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:15.769797 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddefde1c-ad81-4e9c-95aa-7752a3cd7d56 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:15.781841 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5729edd5-bf82-4e3a-9c0a-b4f8d2061b24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:15.783407 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5729edd5-bf82-4e3a-9c0a-b4f8d2061b24 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:15.784283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 140/269] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:44:15 2022] GET /identity/v3/role_assignments?user.id=9ede9378f81a45cc90c51b31386054c2&scope.project.id=08b4540d439e44adb9398aec4130989a&role.id=31d4b1dc082c41f7b6f18c338e7b51df => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:15.785587 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdd639-6450-4cf4-a9e0-08a1b1afe56b None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:15.785874 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdd639-6450-4cf4-a9e0-08a1b1afe56b None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:15.786149 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdd639-6450-4cf4-a9e0-08a1b1afe56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:15.831614 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-1dbdd639-6450-4cf4-a9e0-08a1b1afe56b None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:15.832883 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-1dbdd639-6450-4cf4-a9e0-08a1b1afe56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:15.914494 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 130/270] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:44:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:44:15.915411 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729edd5-bf82-4e3a-9c0a-b4f8d2061b24 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:15.939839 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:15.946827 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:15.991698 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 141/271] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:15.992832 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbdd639-6450-4cf4-a9e0-08a1b1afe56b None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:16.160610 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729edd5-bf82-4e3a-9c0a-b4f8d2061b24 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:16.169488 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbdd639-6450-4cf4-a9e0-08a1b1afe56b None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:16.217720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5a832-9fe2-4d64-8db1-16ae0bed9503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:16.218722 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5a832-9fe2-4d64-8db1-16ae0bed9503 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:16.219174 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5a832-9fe2-4d64-8db1-16ae0bed9503 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:16.219589 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5a832-9fe2-4d64-8db1-16ae0bed9503 None admin] PATH_INFO: `/v3/projects/08b4540d439e44adb9398aec4130989a` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:16.219700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878a8db-276e-44eb-98fb-9c5fac14511b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:16.220317 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878a8db-276e-44eb-98fb-9c5fac14511b None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:16.220584 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878a8db-276e-44eb-98fb-9c5fac14511b None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:16.220857 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878a8db-276e-44eb-98fb-9c5fac14511b None admin] PATH_INFO: `/v3/roles/member` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:16.221373 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a5a832-9fe2-4d64-8db1-16ae0bed9503 None admin] RBAC: Authorizing `identity:get_project(project_id=08b4540d439e44adb9398aec4130989a)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:16.223410 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a5a832-9fe2-4d64-8db1-16ae0bed9503 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:16.225252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 142/272] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:15 2022] GET /identity/v3/projects/08b4540d439e44adb9398aec4130989a => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:16.239658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878a8db-276e-44eb-98fb-9c5fac14511b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:16.247152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-3878a8db-276e-44eb-98fb-9c5fac14511b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:44:16.247152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:16.247152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:16.247152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:16.247152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:16.247152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:16.247152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:16.247152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:16.247152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:16.247152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:16.247152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:16.247152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:16.247152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:16.247152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:16.247152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:16.247152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:16.251204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:16.251204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:16.251204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:16.251204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:16.251204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:16.251204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:16.251204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:16.251204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:16.251204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:16.251204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:16.251204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:16.251204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:16.251204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:16.251204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:16.251204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:16.251204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:16.252205 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:16.252205 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:16.252205 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:16.252205 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:16.252205 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:16.252205 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:16.252205 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:16.252205 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:16.252205 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:16.252205 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:16.252205 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:44:16.252205 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:16.252205 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 131/273] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:16.253161 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5a832-9fe2-4d64-8db1-16ae0bed9503 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:16.268816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878a8db-276e-44eb-98fb-9c5fac14511b None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:16.320453 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:16.380948 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5a832-9fe2-4d64-8db1-16ae0bed9503 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:16.386864 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:16.418433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434fa8a2-2cfa-4744-87c1-1a5e24a595b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:16.419290 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fa8a2-2cfa-4744-87c1-1a5e24a595b9 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:16.419596 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878a8db-276e-44eb-98fb-9c5fac14511b None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:16.419983 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fa8a2-2cfa-4744-87c1-1a5e24a595b9 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:16.420442 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fa8a2-2cfa-4744-87c1-1a5e24a595b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:16.421678 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434fa8a2-2cfa-4744-87c1-1a5e24a595b9 None admin] RBAC: Adding query filter params (role.id=13ca33d694c645d6920930d39dc52a0d, scope.project.id=08b4540d439e44adb9398aec4130989a, user.id=9ede9378f81a45cc90c51b31386054c2) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:16.422136 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434fa8a2-2cfa-4744-87c1-1a5e24a595b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:16.423807 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434fa8a2-2cfa-4744-87c1-1a5e24a595b9 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:16.431124 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 143/274] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:44:16 2022] GET /identity/v3/role_assignments?user.id=9ede9378f81a45cc90c51b31386054c2&scope.project.id=08b4540d439e44adb9398aec4130989a&role.id=13ca33d694c645d6920930d39dc52a0d => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:16.480333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef16f9-c768-4d4b-b89d-3b2bd7900b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:16.481224 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef16f9-c768-4d4b-b89d-3b2bd7900b56 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:16.481649 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef16f9-c768-4d4b-b89d-3b2bd7900b56 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:16.482091 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef16f9-c768-4d4b-b89d-3b2bd7900b56 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:16.483098 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef16f9-c768-4d4b-b89d-3b2bd7900b56 None admin] RBAC: Adding query filter params (name=member) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:16.483781 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef16f9-c768-4d4b-b89d-3b2bd7900b56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:16.485456 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef16f9-c768-4d4b-b89d-3b2bd7900b56 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:16.496006 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 132/275] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 31 10:44:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:16.502664 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434fa8a2-2cfa-4744-87c1-1a5e24a595b9 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:16.511542 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef16f9-c768-4d4b-b89d-3b2bd7900b56 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:16.627244 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef16f9-c768-4d4b-b89d-3b2bd7900b56 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:16.646688 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434fa8a2-2cfa-4744-87c1-1a5e24a595b9 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:16.661927 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:16.674495 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832594b0-e688-4239-86f2-99ad3def7d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:16.680188 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832594b0-e688-4239-86f2-99ad3def7d54 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:16.680188 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832594b0-e688-4239-86f2-99ad3def7d54 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:16.680188 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832594b0-e688-4239-86f2-99ad3def7d54 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:16.680188 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832594b0-e688-4239-86f2-99ad3def7d54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:16.680188 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832594b0-e688-4239-86f2-99ad3def7d54 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:16.686470 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 133/276] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:44:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:16.687285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8416c7-1265-401f-97d8-6782aeab86f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:16.687889 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8416c7-1265-401f-97d8-6782aeab86f4 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:16.688058 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab927d39-43c6-4143-88c9-98783f9e490b None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:16.688232 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8416c7-1265-401f-97d8-6782aeab86f4 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:16.688349 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab927d39-43c6-4143-88c9-98783f9e490b None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:16.688501 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8416c7-1265-401f-97d8-6782aeab86f4 None admin] PATH_INFO: `/v3/users/039465954f9c46e08badce259ad891c8` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:16.688593 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab927d39-43c6-4143-88c9-98783f9e490b None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:16.690272 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 134/277] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:16.690991 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab927d39-43c6-4143-88c9-98783f9e490b None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:16.693826 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:16.707534 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8416c7-1265-401f-97d8-6782aeab86f4 None admin] RBAC: Authorizing `identity:get_user(user_id=039465954f9c46e08badce259ad891c8)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:16.710527 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8416c7-1265-401f-97d8-6782aeab86f4 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:16.712394 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 144/278] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:44:16 2022] GET /identity/v3/users/039465954f9c46e08badce259ad891c8 => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:16.713793 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b338569-64ed-4c3b-b98d-3246b7be3c79 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:16.714507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b338569-64ed-4c3b-b98d-3246b7be3c79 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:16.714940 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b338569-64ed-4c3b-b98d-3246b7be3c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:16.734784 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-1b338569-64ed-4c3b-b98d-3246b7be3c79 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:16.736273 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-1b338569-64ed-4c3b-b98d-3246b7be3c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:16.814163 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 145/279] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:16.816144 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b338569-64ed-4c3b-b98d-3246b7be3c79 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:16.849863 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab927d39-43c6-4143-88c9-98783f9e490b None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:16.898150 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b8a45b-fb81-4168-a9a0-6b214f46376e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:16.899010 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8a45b-fb81-4168-a9a0-6b214f46376e None admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:16.899825 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8a45b-fb81-4168-a9a0-6b214f46376e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:16.900558 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8a45b-fb81-4168-a9a0-6b214f46376e None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:16.901368 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b8a45b-fb81-4168-a9a0-6b214f46376e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:16.903748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b8a45b-fb81-4168-a9a0-6b214f46376e None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:16.921568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:16.924788 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b338569-64ed-4c3b-b98d-3246b7be3c79 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:16.962449 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08debebf-2304-4881-b628-7d0956627af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:16.963095 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08debebf-2304-4881-b628-7d0956627af0 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:16.963374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08debebf-2304-4881-b628-7d0956627af0 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:16.963645 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08debebf-2304-4881-b628-7d0956627af0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:16.964374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08debebf-2304-4881-b628-7d0956627af0 None admin] RBAC: Adding query filter params (role.id=2278da3c52644a819d210b5f9c20b6c1, user.id=039465954f9c46e08badce259ad891c8, scope.system=all) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:16.964729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08debebf-2304-4881-b628-7d0956627af0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:16.966463 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08debebf-2304-4881-b628-7d0956627af0 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:16.971809 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 146/280] 10.222.0.16 () {56 vars in 1308 bytes} [Wed Aug 31 10:44:16 2022] GET /identity/v3/role_assignments?user.id=039465954f9c46e08badce259ad891c8&scope.system=all&role.id=2278da3c52644a819d210b5f9c20b6c1 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:17.016662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 135/281] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 31 10:44:16 2022] POST /identity/v3/users => generated 293 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:44:17.089225 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08debebf-2304-4881-b628-7d0956627af0 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:17.215688 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08debebf-2304-4881-b628-7d0956627af0 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:17.252659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b5472-424b-4cde-843f-905be4337739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:17.253287 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b5472-424b-4cde-843f-905be4337739 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:17.253564 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b5472-424b-4cde-843f-905be4337739 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:17.253829 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b5472-424b-4cde-843f-905be4337739 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:17.265550 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b5472-424b-4cde-843f-905be4337739 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:17.270519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-808b5472-424b-4cde-843f-905be4337739 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:44:17.270519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:17.270519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:17.270519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:17.270519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:17.270519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:17.270519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:17.270519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:17.270519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:17.270519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:17.270519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:17.270519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:17.270519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:17.270519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:17.270519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:17.270519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:17.271405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:17.271405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:17.271405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:17.271405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:17.271405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:17.271405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:17.271405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:17.271405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:17.271405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:17.271405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:17.271405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:17.271405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:17.271405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:17.271405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:17.271405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:17.271405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:17.272170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:17.272170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:17.272170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:17.272170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:17.272170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:17.272170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:17.272170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:17.272170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:17.272170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:17.272170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:17.272170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:44:17.272170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:17.272170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 147/282] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:17.277766 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b8a45b-fb81-4168-a9a0-6b214f46376e None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:17.429915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:17.478879 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b8a45b-fb81-4168-a9a0-6b214f46376e None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:17.521928 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d9bea-982d-417f-8fc7-36f4dfa69d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:17.522642 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d9bea-982d-417f-8fc7-36f4dfa69d6a None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:17.522922 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d9bea-982d-417f-8fc7-36f4dfa69d6a None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:17.528198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d9bea-982d-417f-8fc7-36f4dfa69d6a None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:17.528198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d9bea-982d-417f-8fc7-36f4dfa69d6a None admin] RBAC: Adding query filter params (name=reader) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:17.528198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d9bea-982d-417f-8fc7-36f4dfa69d6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:17.528198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d9bea-982d-417f-8fc7-36f4dfa69d6a None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:17.535136 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 136/283] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 31 10:44:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:17.540938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b5472-424b-4cde-843f-905be4337739 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:17.563981 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e76174-e9ac-4a14-8197-c818dd00ade2 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:17.564279 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e76174-e9ac-4a14-8197-c818dd00ade2 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:17.564532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e76174-e9ac-4a14-8197-c818dd00ade2 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:17.567248 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 137/284] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:17.571347 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b587f1-cc14-4a9c-b68e-c0d66de05124 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:17.571645 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b587f1-cc14-4a9c-b68e-c0d66de05124 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:17.571982 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b587f1-cc14-4a9c-b68e-c0d66de05124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:17.592078 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-34b587f1-cc14-4a9c-b68e-c0d66de05124 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:17.593398 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-34b587f1-cc14-4a9c-b68e-c0d66de05124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:17.600670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:17.679921 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 138/285] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:44:17.733370 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b5472-424b-4cde-843f-905be4337739 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:17.788981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9f7a1-7825-4bfd-b4b8-50057158eb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:17.789666 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9f7a1-7825-4bfd-b4b8-50057158eb8a None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:17.789940 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9f7a1-7825-4bfd-b4b8-50057158eb8a None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:17.790206 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9f7a1-7825-4bfd-b4b8-50057158eb8a None admin] PATH_INFO: `/v3/users/be08a1a635ea4edf8ac4f73b7eefa049` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:17.791375 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9f7a1-7825-4bfd-b4b8-50057158eb8a None admin] RBAC: Authorizing `identity:get_user(user_id=be08a1a635ea4edf8ac4f73b7eefa049)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:17.792831 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9f7a1-7825-4bfd-b4b8-50057158eb8a None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:17.794930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 148/286] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:44:17 2022] GET /identity/v3/users/be08a1a635ea4edf8ac4f73b7eefa049 => generated 297 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:17.800981 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b587f1-cc14-4a9c-b68e-c0d66de05124 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:17.868678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:17.931974 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b587f1-cc14-4a9c-b68e-c0d66de05124 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:17.974289 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98ceb9-0995-4e51-a97c-e1eefa4b10ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:17.974964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98ceb9-0995-4e51-a97c-e1eefa4b10ea None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:17.975252 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98ceb9-0995-4e51-a97c-e1eefa4b10ea None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:17.975517 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98ceb9-0995-4e51-a97c-e1eefa4b10ea None admin] PATH_INFO: `/v3/projects/08b4540d439e44adb9398aec4130989a` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:17.976697 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c98ceb9-0995-4e51-a97c-e1eefa4b10ea None admin] RBAC: Authorizing `identity:get_project(project_id=08b4540d439e44adb9398aec4130989a)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:17.978147 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c98ceb9-0995-4e51-a97c-e1eefa4b10ea None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:17.980103 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 139/287] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:17 2022] GET /identity/v3/projects/08b4540d439e44adb9398aec4130989a => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:17.985870 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9f7a1-7825-4bfd-b4b8-50057158eb8a None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:18.036388 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98ceb9-0995-4e51-a97c-e1eefa4b10ea None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:18.125077 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:18.135261 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9f7a1-7825-4bfd-b4b8-50057158eb8a None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:18.174009 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19205728-9686-44cc-ab92-a8654751b7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:18.174855 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19205728-9686-44cc-ab92-a8654751b7df None admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:18.175296 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19205728-9686-44cc-ab92-a8654751b7df None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:18.175730 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19205728-9686-44cc-ab92-a8654751b7df None admin] PATH_INFO: `/v3/projects/08b4540d439e44adb9398aec4130989a/users/be08a1a635ea4edf8ac4f73b7eefa049/roles/5585e95a315947d1ad314d7978b46e3a` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:18.176342 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98ceb9-0995-4e51-a97c-e1eefa4b10ea None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:18.177977 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19205728-9686-44cc-ab92-a8654751b7df None admin] RBAC: Authorizing `identity:create_grant(project_id=08b4540d439e44adb9398aec4130989a, user_id=be08a1a635ea4edf8ac4f73b7eefa049, role_id=5585e95a315947d1ad314d7978b46e3a)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:18.179719 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19205728-9686-44cc-ab92-a8654751b7df None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:18.195713 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:18.214935 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441414a-661e-4c55-95b8-7ad6005fd3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:18.215592 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441414a-661e-4c55-95b8-7ad6005fd3a3 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:18.215865 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441414a-661e-4c55-95b8-7ad6005fd3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:18.216134 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441414a-661e-4c55-95b8-7ad6005fd3a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:18.227231 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441414a-661e-4c55-95b8-7ad6005fd3a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:18.232285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-e441414a-661e-4c55-95b8-7ad6005fd3a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:44:18.232285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:18.232285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:18.232285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:18.232285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:18.232285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:18.232285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:18.232285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:18.232285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:18.232285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:18.232285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:18.232285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:18.232285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:18.232285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:18.232285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:18.232285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:18.233720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:18.233720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:18.233720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:18.233720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:18.233720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:18.233720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:18.233720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:18.233720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:18.233720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:18.233720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:18.233720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:18.233720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:18.233720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:18.233720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:18.233720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:18.233720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:18.234757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:18.234757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:18.234757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:18.234757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:18.234757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:18.234757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:18.234757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:18.234757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:18.234757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:18.234757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:18.234757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:44:18.234757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:18.234757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 140/288] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:44:18.240657 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441414a-661e-4c55-95b8-7ad6005fd3a3 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:18.266428 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 149/289] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 31 10:44:17 2022] PUT /identity/v3/projects/08b4540d439e44adb9398aec4130989a/users/be08a1a635ea4edf8ac4f73b7eefa049/roles/5585e95a315947d1ad314d7978b46e3a => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:44:18.371424 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441414a-661e-4c55-95b8-7ad6005fd3a3 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:18.412647 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f283dc5-2a7e-4d22-b64e-c7d09ce9a812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:18.413280 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f283dc5-2a7e-4d22-b64e-c7d09ce9a812 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:18.413554 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f283dc5-2a7e-4d22-b64e-c7d09ce9a812 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:18.413825 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f283dc5-2a7e-4d22-b64e-c7d09ce9a812 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:18.414535 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f283dc5-2a7e-4d22-b64e-c7d09ce9a812 None admin] RBAC: Adding query filter params (name=member) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:18.414976 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f283dc5-2a7e-4d22-b64e-c7d09ce9a812 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:18.416375 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f283dc5-2a7e-4d22-b64e-c7d09ce9a812 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:18.426814 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 141/290] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 31 10:44:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:18.432548 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19205728-9686-44cc-ab92-a8654751b7df None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:18.567353 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19205728-9686-44cc-ab92-a8654751b7df None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:18.606064 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134e90c-06c8-45a4-bd0b-e2daf94b3640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:18.606792 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134e90c-06c8-45a4-bd0b-e2daf94b3640 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:18.607072 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134e90c-06c8-45a4-bd0b-e2daf94b3640 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:18.607341 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134e90c-06c8-45a4-bd0b-e2daf94b3640 None admin] PATH_INFO: `/v3/users/d7759a1e2e334613860406db8bb5efd9` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:18.608636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e134e90c-06c8-45a4-bd0b-e2daf94b3640 None admin] RBAC: Authorizing `identity:get_user(user_id=d7759a1e2e334613860406db8bb5efd9)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:18.610079 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e134e90c-06c8-45a4-bd0b-e2daf94b3640 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:18.611777 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 150/291] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:44:18 2022] GET /identity/v3/users/d7759a1e2e334613860406db8bb5efd9 => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:18.617877 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f283dc5-2a7e-4d22-b64e-c7d09ce9a812 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:18.695926 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:18.778216 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f283dc5-2a7e-4d22-b64e-c7d09ce9a812 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:18.824200 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a185b9-0bd9-4845-b740-eeecf3e5664e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:18.824744 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a185b9-0bd9-4845-b740-eeecf3e5664e None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:18.825025 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a185b9-0bd9-4845-b740-eeecf3e5664e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:18.825299 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a185b9-0bd9-4845-b740-eeecf3e5664e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:18.922668 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a185b9-0bd9-4845-b740-eeecf3e5664e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:18.924130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a185b9-0bd9-4845-b740-eeecf3e5664e None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:18.976174 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:19.069829 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 142/292] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:44:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:44:19.075843 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134e90c-06c8-45a4-bd0b-e2daf94b3640 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:19.122885 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:19.239111 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134e90c-06c8-45a4-bd0b-e2daf94b3640 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:19.288390 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02522f6-9a99-466a-8e40-5df55ad4d180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:19.289030 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02522f6-9a99-466a-8e40-5df55ad4d180 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:19.289305 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02522f6-9a99-466a-8e40-5df55ad4d180 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:19.289573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02522f6-9a99-466a-8e40-5df55ad4d180 None admin] PATH_INFO: `/v3/projects/08b4540d439e44adb9398aec4130989a` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:19.290721 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02522f6-9a99-466a-8e40-5df55ad4d180 None admin] RBAC: Authorizing `identity:get_project(project_id=08b4540d439e44adb9398aec4130989a)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:19.292230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02522f6-9a99-466a-8e40-5df55ad4d180 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:19.294206 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 151/293] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:19 2022] GET /identity/v3/projects/08b4540d439e44adb9398aec4130989a => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:19.300350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a185b9-0bd9-4845-b740-eeecf3e5664e None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:19.377811 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d395a0f-9639-445f-a2e0-ad0878966269 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:19.377997 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d395a0f-9639-445f-a2e0-ad0878966269 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:19.378267 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d395a0f-9639-445f-a2e0-ad0878966269 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:19.379635 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 152/294] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:44:19.388177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f755f-6de8-454e-8932-7de031f4356f None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:19.388454 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f755f-6de8-454e-8932-7de031f4356f None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:19.388822 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f755f-6de8-454e-8932-7de031f4356f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:19.412204 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-268f755f-6de8-454e-8932-7de031f4356f None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:19.413398 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-268f755f-6de8-454e-8932-7de031f4356f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:19.456868 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a185b9-0bd9-4845-b740-eeecf3e5664e None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:19.502446 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b06e8bc-396b-4550-8982-1cd3080574da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:19.503415 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06e8bc-396b-4550-8982-1cd3080574da None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:19.503887 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06e8bc-396b-4550-8982-1cd3080574da None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:19.504401 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06e8bc-396b-4550-8982-1cd3080574da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:19.505365 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b06e8bc-396b-4550-8982-1cd3080574da None admin] RBAC: Adding query filter params (role.id=2278da3c52644a819d210b5f9c20b6c1, scope.project.id=08b4540d439e44adb9398aec4130989a, user.id=d7759a1e2e334613860406db8bb5efd9) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:19.505864 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b06e8bc-396b-4550-8982-1cd3080574da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:19.507607 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b06e8bc-396b-4550-8982-1cd3080574da None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:19.520092 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 153/295] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:44:19.526971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 143/296] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:44:19 2022] GET /identity/v3/role_assignments?user.id=d7759a1e2e334613860406db8bb5efd9&scope.project.id=08b4540d439e44adb9398aec4130989a&role.id=2278da3c52644a819d210b5f9c20b6c1 => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:19.603507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60fcd88-ecc2-451c-ad0e-9e5b8d841f68 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:19.603792 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60fcd88-ecc2-451c-ad0e-9e5b8d841f68 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:19.604237 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60fcd88-ecc2-451c-ad0e-9e5b8d841f68 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:19.606360 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 154/297] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:19.612365 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b9009-7c16-4663-9f63-792473984562 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:19.612799 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b9009-7c16-4663-9f63-792473984562 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:19.613176 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b9009-7c16-4663-9f63-792473984562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:19.623779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:19.637952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-c76b9009-7c16-4663-9f63-792473984562 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:19.637952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-c76b9009-7c16-4663-9f63-792473984562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:19.729075 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 144/298] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:19.812343 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3e507-8fa4-47ae-8bce-967f4e136899 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:19.812848 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3e507-8fa4-47ae-8bce-967f4e136899 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:19.813261 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3e507-8fa4-47ae-8bce-967f4e136899 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:19.815206 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 155/299] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:19.820962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867973a-78d0-4444-8675-892910701382 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:19.821556 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867973a-78d0-4444-8675-892910701382 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:19.822031 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867973a-78d0-4444-8675-892910701382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:19.839405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3e507-8fa4-47ae-8bce-967f4e136899 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:19.843160 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-5867973a-78d0-4444-8675-892910701382 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:19.844422 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-5867973a-78d0-4444-8675-892910701382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:19.862381 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:19.923291 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 145/300] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:19.956811 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3e507-8fa4-47ae-8bce-967f4e136899 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:19.996415 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e7bc7-bb26-4757-8a01-0bf9241af60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:19.997081 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e7bc7-bb26-4757-8a01-0bf9241af60a None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:19.997357 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e7bc7-bb26-4757-8a01-0bf9241af60a None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:19.997632 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e7bc7-bb26-4757-8a01-0bf9241af60a None admin] PATH_INFO: `/v3/roles/member` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:20.009977 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e7bc7-bb26-4757-8a01-0bf9241af60a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:20.015478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-114e7bc7-bb26-4757-8a01-0bf9241af60a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:44:20.015478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:20.015478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:20.015478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:20.015478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:20.015478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:20.015478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:20.015478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:20.015478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:20.015478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:20.015478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:20.015478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:20.015478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:20.015478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:20.015478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:20.015478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:20.016367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:20.016367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:20.016367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:20.016367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:20.016367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:20.016367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:20.016367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:20.016367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:20.016367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:20.016367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:20.016367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:20.016367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:20.016367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:20.016367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:20.016367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:20.016367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:20.017155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:20.017155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:20.017155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:20.017155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:20.017155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:20.017155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:20.017155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:20.017155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:20.017155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:20.017155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:20.017155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:44:20.017155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:20.021508 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 156/301] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:20.023589 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867973a-78d0-4444-8675-892910701382 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:20.031473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e7bc7-bb26-4757-8a01-0bf9241af60a None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:20.112415 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:20.137744 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:20.165622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867973a-78d0-4444-8675-892910701382 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:20.171998 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e7bc7-bb26-4757-8a01-0bf9241af60a None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:20.203267 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc187bb-aed2-494d-9b6c-e1c0e7a863c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:20.204288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc187bb-aed2-494d-9b6c-e1c0e7a863c6 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:20.205017 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc187bb-aed2-494d-9b6c-e1c0e7a863c6 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:20.205511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc187bb-aed2-494d-9b6c-e1c0e7a863c6 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:20.206410 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc187bb-aed2-494d-9b6c-e1c0e7a863c6 None admin] RBAC: Adding query filter params (name=member) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:20.206940 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc187bb-aed2-494d-9b6c-e1c0e7a863c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:20.208676 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc187bb-aed2-494d-9b6c-e1c0e7a863c6 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:20.218052 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53da6f-1fcb-42ef-9390-5d576d20108d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:20.218667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 146/302] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 31 10:44:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:44:20.219370 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc187bb-aed2-494d-9b6c-e1c0e7a863c6 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:20.222548 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53da6f-1fcb-42ef-9390-5d576d20108d None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:20.222866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53da6f-1fcb-42ef-9390-5d576d20108d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:20.223142 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53da6f-1fcb-42ef-9390-5d576d20108d None admin] PATH_INFO: `/v3/domains/default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:20.224374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53da6f-1fcb-42ef-9390-5d576d20108d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:20.225893 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53da6f-1fcb-42ef-9390-5d576d20108d None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:20.227530 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 157/303] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:44:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:20.228304 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53da6f-1fcb-42ef-9390-5d576d20108d None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:20.337846 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc187bb-aed2-494d-9b6c-e1c0e7a863c6 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:20.339645 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53da6f-1fcb-42ef-9390-5d576d20108d None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:20.368444 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:20.376123 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68996953-e456-4985-a177-92bb783948be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:20.376979 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68996953-e456-4985-a177-92bb783948be None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:20.377259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68996953-e456-4985-a177-92bb783948be None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:20.377531 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68996953-e456-4985-a177-92bb783948be None admin] PATH_INFO: `/v3/users/039465954f9c46e08badce259ad891c8` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:20.377628 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506e105-e62d-434a-8c7c-5006e5157516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:20.378239 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506e105-e62d-434a-8c7c-5006e5157516 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:20.378566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506e105-e62d-434a-8c7c-5006e5157516 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:20.378832 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68996953-e456-4985-a177-92bb783948be None admin] RBAC: Authorizing `identity:get_user(user_id=039465954f9c46e08badce259ad891c8)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:20.383545 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506e105-e62d-434a-8c7c-5006e5157516 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:20.383650 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68996953-e456-4985-a177-92bb783948be None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:20.383650 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 158/304] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:44:20 2022] GET /identity/v3/users/039465954f9c46e08badce259ad891c8 => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:20.383650 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68996953-e456-4985-a177-92bb783948be None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:20.393703 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506e105-e62d-434a-8c7c-5006e5157516 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:20.399311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-5506e105-e62d-434a-8c7c-5006e5157516 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:44:20.399311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:20.399311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:20.399311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:20.399311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:20.399311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:20.399311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:20.399311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:20.399311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:20.399311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:20.399311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:20.399311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:20.399311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:20.399311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:20.399311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:20.399311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:20.400156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:20.400156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:20.400156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:20.400156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:20.400156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:20.400156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:20.400156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:20.400156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:20.400156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:20.400156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:20.400156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:20.400156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:20.400156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:20.400156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:20.400156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:20.400156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:20.400913 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:20.400913 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:20.400913 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:20.400913 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:20.400913 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:20.400913 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:20.400913 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:20.400913 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:20.400913 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:20.400913 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:20.400913 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:44:20.400913 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:20.400913 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 147/305] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:20.401530 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506e105-e62d-434a-8c7c-5006e5157516 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:20.408756 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:20.499937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68996953-e456-4985-a177-92bb783948be None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:20.518415 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506e105-e62d-434a-8c7c-5006e5157516 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:20.537161 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea665bf-ad2d-4909-87ba-abd52a7b647b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:20.537825 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea665bf-ad2d-4909-87ba-abd52a7b647b None admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:20.538103 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea665bf-ad2d-4909-87ba-abd52a7b647b None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:20.538363 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea665bf-ad2d-4909-87ba-abd52a7b647b None admin] PATH_INFO: `/v3/groups` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:20.539013 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea665bf-ad2d-4909-87ba-abd52a7b647b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:20.540443 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea665bf-ad2d-4909-87ba-abd52a7b647b None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:20.554891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad2067-330a-4c7b-ab4e-c747b947c682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:20.555618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad2067-330a-4c7b-ab4e-c747b947c682 None admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:20.556584 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad2067-330a-4c7b-ab4e-c747b947c682 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:20.556584 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad2067-330a-4c7b-ab4e-c747b947c682 None admin] PATH_INFO: `/v3/system/users/039465954f9c46e08badce259ad891c8/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:20.557656 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ad2067-330a-4c7b-ab4e-c747b947c682 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=039465954f9c46e08badce259ad891c8, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:20.559057 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ad2067-330a-4c7b-ab4e-c747b947c682 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:20.603940 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 159/306] 10.222.0.16 () {60 vars in 1134 bytes} [Wed Aug 31 10:44:20 2022] POST /identity/v3/groups => generated 229 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:44:20.604835 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea665bf-ad2d-4909-87ba-abd52a7b647b None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:20.622038 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 148/307] 10.222.0.16 () {58 vars in 1334 bytes} [Wed Aug 31 10:44:20 2022] PUT /identity/v3/system/users/039465954f9c46e08badce259ad891c8/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:44:20.726284 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:20.758636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea665bf-ad2d-4909-87ba-abd52a7b647b None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:20.804349 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83070e4f-b525-4bed-8795-742aae4c66d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:20.805080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83070e4f-b525-4bed-8795-742aae4c66d3 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:20.805359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83070e4f-b525-4bed-8795-742aae4c66d3 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:20.805689 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83070e4f-b525-4bed-8795-742aae4c66d3 None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:20.806376 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83070e4f-b525-4bed-8795-742aae4c66d3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:20.806705 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83070e4f-b525-4bed-8795-742aae4c66d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:20.808116 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83070e4f-b525-4bed-8795-742aae4c66d3 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:20.818909 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 160/308] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 31 10:44:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:20.824427 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad2067-330a-4c7b-ab4e-c747b947c682 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:20.887892 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:20.898130 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60af1dd-cac6-465d-9160-147558dffb16 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:20.898453 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60af1dd-cac6-465d-9160-147558dffb16 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:20.898734 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60af1dd-cac6-465d-9160-147558dffb16 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:20.900316 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 161/309] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:44:20.905678 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8a01e-2f0a-41d6-9239-865bcafc5e97 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:20.905954 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8a01e-2f0a-41d6-9239-865bcafc5e97 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:20.906221 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8a01e-2f0a-41d6-9239-865bcafc5e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:20.927750 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-b1a8a01e-2f0a-41d6-9239-865bcafc5e97 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:20.928976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-b1a8a01e-2f0a-41d6-9239-865bcafc5e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:20.942832 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad2067-330a-4c7b-ab4e-c747b947c682 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:20.987864 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114e10a-2015-4fb5-a9aa-0c255881389d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:20.988949 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114e10a-2015-4fb5-a9aa-0c255881389d None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:20.989449 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114e10a-2015-4fb5-a9aa-0c255881389d None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:20.989923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114e10a-2015-4fb5-a9aa-0c255881389d None admin] PATH_INFO: `/v3/users/346505f36a3f43749e066c171b35ca66` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:21.006480 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114e10a-2015-4fb5-a9aa-0c255881389d None admin] RBAC: Authorizing `identity:get_user(user_id=346505f36a3f43749e066c171b35ca66)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:21.008470 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114e10a-2015-4fb5-a9aa-0c255881389d None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:21.010734 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 149/310] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:44:20 2022] GET /identity/v3/users/346505f36a3f43749e066c171b35ca66 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:21.018449 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114e10a-2015-4fb5-a9aa-0c255881389d None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:21.038660 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 162/311] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:21.142127 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114e10a-2015-4fb5-a9aa-0c255881389d None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:21.153428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:21.183106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a14c0-a943-4f26-881a-c709cc1f8944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:21.183720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a14c0-a943-4f26-881a-c709cc1f8944 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:21.184100 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a14c0-a943-4f26-881a-c709cc1f8944 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:21.184377 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a14c0-a943-4f26-881a-c709cc1f8944 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:21.185250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a14c0-a943-4f26-881a-c709cc1f8944 None admin] RBAC: Adding query filter params (role.id=5585e95a315947d1ad314d7978b46e3a, user.id=346505f36a3f43749e066c171b35ca66, scope.system=all) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:21.186072 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a14c0-a943-4f26-881a-c709cc1f8944 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:21.187803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a14c0-a943-4f26-881a-c709cc1f8944 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:21.193429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 150/312] 10.222.0.16 () {56 vars in 1308 bytes} [Wed Aug 31 10:44:21 2022] GET /identity/v3/role_assignments?user.id=346505f36a3f43749e066c171b35ca66&scope.system=all&role.id=5585e95a315947d1ad314d7978b46e3a => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:44:21.344349 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8a01e-2f0a-41d6-9239-865bcafc5e97 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:21.415045 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:21.450379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8a01e-2f0a-41d6-9239-865bcafc5e97 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:21.491232 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bb57f-ff73-4e6c-8eb6-18517b4889b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:21.492205 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bb57f-ff73-4e6c-8eb6-18517b4889b8 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:21.492674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bb57f-ff73-4e6c-8eb6-18517b4889b8 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:21.493135 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bb57f-ff73-4e6c-8eb6-18517b4889b8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:21.503893 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3bb57f-ff73-4e6c-8eb6-18517b4889b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:21.509375 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-ca3bb57f-ff73-4e6c-8eb6-18517b4889b8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:44:21.509375 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:21.509375 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:21.509375 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:21.509375 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:21.509375 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:21.509375 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:21.509375 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:21.509375 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:21.509375 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:21.509375 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:21.509375 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:21.509375 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:21.509375 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:21.509375 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:21.509375 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:21.510231 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:21.510231 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:21.510231 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:21.510231 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:21.510231 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:21.510231 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:21.510231 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:21.510231 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:21.510231 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:21.510231 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:21.510231 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:21.510231 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:21.510231 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:21.510231 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:21.510231 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:21.510231 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:21.511040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:21.511040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:21.511040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:21.511040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:21.511040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:21.511040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:21.511040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:21.511040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:21.511040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:21.511040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:21.511040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:44:21.511040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:21.513238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 163/313] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:21.519671 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a14c0-a943-4f26-881a-c709cc1f8944 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:21.635876 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a14c0-a943-4f26-881a-c709cc1f8944 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:21.672661 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94187664-56d7-492f-befe-8e6c879f7bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:21.673288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94187664-56d7-492f-befe-8e6c879f7bb7 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:21.673563 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94187664-56d7-492f-befe-8e6c879f7bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:21.673834 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94187664-56d7-492f-befe-8e6c879f7bb7 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:21.674592 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94187664-56d7-492f-befe-8e6c879f7bb7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:21.674890 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94187664-56d7-492f-befe-8e6c879f7bb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:21.676268 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94187664-56d7-492f-befe-8e6c879f7bb7 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:21.687342 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 151/314] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 31 10:44:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:21.692890 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bb57f-ff73-4e6c-8eb6-18517b4889b8 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:21.799932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bb57f-ff73-4e6c-8eb6-18517b4889b8 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:21.830234 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:21.839166 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf66336-aaa5-4515-9ce0-200dfc964ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:21.840005 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf66336-aaa5-4515-9ce0-200dfc964ec7 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:21.841604 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf66336-aaa5-4515-9ce0-200dfc964ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:21.841604 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf66336-aaa5-4515-9ce0-200dfc964ec7 None admin] PATH_INFO: `/v3/users/be08a1a635ea4edf8ac4f73b7eefa049` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:21.842248 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf66336-aaa5-4515-9ce0-200dfc964ec7 None admin] RBAC: Authorizing `identity:get_user(user_id=be08a1a635ea4edf8ac4f73b7eefa049)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:21.843906 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf66336-aaa5-4515-9ce0-200dfc964ec7 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:21.846034 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 164/315] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:44:21 2022] GET /identity/v3/users/be08a1a635ea4edf8ac4f73b7eefa049 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:21.851950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94187664-56d7-492f-befe-8e6c879f7bb7 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:21.891461 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:21.960203 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94187664-56d7-492f-befe-8e6c879f7bb7 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:22.000240 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e617b-150f-43ef-a384-654130b76f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:22.000852 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e617b-150f-43ef-a384-654130b76f7d None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:22.001129 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e617b-150f-43ef-a384-654130b76f7d None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:22.001400 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e617b-150f-43ef-a384-654130b76f7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:22.071230 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e617b-150f-43ef-a384-654130b76f7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:22.072686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e617b-150f-43ef-a384-654130b76f7d None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:22.114415 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:22.176812 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 152/316] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:44:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:44:22.186971 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf66336-aaa5-4515-9ce0-200dfc964ec7 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:22.303198 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf66336-aaa5-4515-9ce0-200dfc964ec7 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:22.340961 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38764c-dcb1-4d35-b403-c365ba813411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:22.342724 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38764c-dcb1-4d35-b403-c365ba813411 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:22.343239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38764c-dcb1-4d35-b403-c365ba813411 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:22.343724 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38764c-dcb1-4d35-b403-c365ba813411 None admin] PATH_INFO: `/v3/projects/08b4540d439e44adb9398aec4130989a` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:22.345106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c38764c-dcb1-4d35-b403-c365ba813411 None admin] RBAC: Authorizing `identity:get_project(project_id=08b4540d439e44adb9398aec4130989a)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:22.347198 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c38764c-dcb1-4d35-b403-c365ba813411 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:22.349367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 165/317] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:22 2022] GET /identity/v3/projects/08b4540d439e44adb9398aec4130989a => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:22.355839 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e617b-150f-43ef-a384-654130b76f7d None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:22.462948 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e617b-150f-43ef-a384-654130b76f7d None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:22.499832 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85e6c6-079a-4850-af7f-71d6f5caee01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:22.500440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85e6c6-079a-4850-af7f-71d6f5caee01 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:22.500718 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85e6c6-079a-4850-af7f-71d6f5caee01 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:22.500987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85e6c6-079a-4850-af7f-71d6f5caee01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:22.501708 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b85e6c6-079a-4850-af7f-71d6f5caee01 None admin] RBAC: Adding query filter params (role.id=5585e95a315947d1ad314d7978b46e3a, scope.project.id=08b4540d439e44adb9398aec4130989a, user.id=be08a1a635ea4edf8ac4f73b7eefa049) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:22.502012 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b85e6c6-079a-4850-af7f-71d6f5caee01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:22.503466 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b85e6c6-079a-4850-af7f-71d6f5caee01 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:22.505418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:22.510825 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 153/318] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:44:22 2022] GET /identity/v3/role_assignments?user.id=be08a1a635ea4edf8ac4f73b7eefa049&scope.project.id=08b4540d439e44adb9398aec4130989a&role.id=5585e95a315947d1ad314d7978b46e3a => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:22.708143 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1d975-6fd1-45da-bed0-ca06ddbf4a86 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:22.708421 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1d975-6fd1-45da-bed0-ca06ddbf4a86 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:22.708854 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1d975-6fd1-45da-bed0-ca06ddbf4a86 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:22.710619 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 166/319] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:22.716123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58399f4e-e59c-4c5d-b990-7a67189953ab None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:22.716675 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58399f4e-e59c-4c5d-b990-7a67189953ab None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:22.717180 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58399f4e-e59c-4c5d-b990-7a67189953ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:22.737695 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9dbae4-6840-48fb-9fdf-ef680e846209 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:22.738001 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9dbae4-6840-48fb-9fdf-ef680e846209 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:22.738236 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9dbae4-6840-48fb-9fdf-ef680e846209 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:22.739763 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 167/320] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:22.739864 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-58399f4e-e59c-4c5d-b990-7a67189953ab None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:22.741066 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-58399f4e-e59c-4c5d-b990-7a67189953ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:22.746092 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774efaef-d86f-4c5f-b982-52e33a16f539 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:22.746379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774efaef-d86f-4c5f-b982-52e33a16f539 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:22.746693 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774efaef-d86f-4c5f-b982-52e33a16f539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:22.766916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-774efaef-d86f-4c5f-b982-52e33a16f539 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:22.769199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-774efaef-d86f-4c5f-b982-52e33a16f539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:22.813349 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 154/321] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:22.836719 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:22.848298 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 168/322] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:44:23.086609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58399f4e-e59c-4c5d-b990-7a67189953ab None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:23.116474 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774efaef-d86f-4c5f-b982-52e33a16f539 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:23.196863 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58399f4e-e59c-4c5d-b990-7a67189953ab None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:23.223130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:23.224509 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774efaef-d86f-4c5f-b982-52e33a16f539 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:23.230627 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86bfbc-432e-4d37-a66f-87f219a20e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:23.233118 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86bfbc-432e-4d37-a66f-87f219a20e85 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:23.233118 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86bfbc-432e-4d37-a66f-87f219a20e85 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:23.233118 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86bfbc-432e-4d37-a66f-87f219a20e85 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:23.242134 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad86bfbc-432e-4d37-a66f-87f219a20e85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:23.247282 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-ad86bfbc-432e-4d37-a66f-87f219a20e85 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:44:23.247282 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:23.247282 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:23.247282 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:23.247282 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:23.247282 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:23.247282 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:23.247282 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:23.247282 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:23.247282 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:23.247282 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:23.247282 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:23.247282 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:23.247282 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:23.247282 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:23.247282 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:23.248365 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:23.248365 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:23.248365 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:23.248365 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:23.248365 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:23.248365 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:23.248365 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:23.248365 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:23.248365 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:23.248365 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:23.248365 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:23.248365 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:23.248365 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:23.248365 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:23.248365 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:23.248365 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:23.249177 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:23.249177 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:23.249177 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:23.249177 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:23.249177 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:23.249177 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:23.249177 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:23.249177 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:23.249177 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:23.249177 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:23.249177 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:44:23.249177 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:23.249177 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 155/323] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:23.250230 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6392d-0bea-46f8-8913-cb6318533732 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:23.250533 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6392d-0bea-46f8-8913-cb6318533732 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:23.250884 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6392d-0bea-46f8-8913-cb6318533732 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:23.256205 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 156/324] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:23.256205 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e6392d-0bea-46f8-8913-cb6318533732 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:23.260855 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e28c46-ef1a-4216-8df9-795d97b15c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:23.261476 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e28c46-ef1a-4216-8df9-795d97b15c79 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:23.261749 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e28c46-ef1a-4216-8df9-795d97b15c79 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:23.262012 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e28c46-ef1a-4216-8df9-795d97b15c79 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:23.263214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e28c46-ef1a-4216-8df9-795d97b15c79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:23.264901 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e28c46-ef1a-4216-8df9-795d97b15c79 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:23.266472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 169/325] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:44:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:44:23.267561 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d88c8-c6d0-44bc-ace5-f3f039cb9219 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:23.267863 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d88c8-c6d0-44bc-ace5-f3f039cb9219 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:23.268128 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d88c8-c6d0-44bc-ace5-f3f039cb9219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:23.286851 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-0a8d88c8-c6d0-44bc-ace5-f3f039cb9219 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:23.288017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-0a8d88c8-c6d0-44bc-ace5-f3f039cb9219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:23.344252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:23.355233 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 170/326] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:23.356146 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d88c8-c6d0-44bc-ace5-f3f039cb9219 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:23.363091 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e6392d-0bea-46f8-8913-cb6318533732 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:23.399547 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baca1f3-6ad7-474d-be76-d601c8da5cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:23.400148 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baca1f3-6ad7-474d-be76-d601c8da5cc7 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:23.400418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baca1f3-6ad7-474d-be76-d601c8da5cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:23.400693 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baca1f3-6ad7-474d-be76-d601c8da5cc7 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:23.401346 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baca1f3-6ad7-474d-be76-d601c8da5cc7 None admin] RBAC: Adding query filter params (name=member) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:23.401642 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baca1f3-6ad7-474d-be76-d601c8da5cc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:23.403075 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baca1f3-6ad7-474d-be76-d601c8da5cc7 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:23.416436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 157/327] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 31 10:44:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:23.417404 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baca1f3-6ad7-474d-be76-d601c8da5cc7 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:23.467947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d88c8-c6d0-44bc-ace5-f3f039cb9219 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:23.470585 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:23.505683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643946f-cfb3-4d13-9313-2622b900c472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:23.506305 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643946f-cfb3-4d13-9313-2622b900c472 None admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:23.506605 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643946f-cfb3-4d13-9313-2622b900c472 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:23.506874 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643946f-cfb3-4d13-9313-2622b900c472 None admin] PATH_INFO: `/v3/groups` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:23.507499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643946f-cfb3-4d13-9313-2622b900c472 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:23.508930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643946f-cfb3-4d13-9313-2622b900c472 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:23.521041 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baca1f3-6ad7-474d-be76-d601c8da5cc7 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:23.554876 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232746f-a5c1-43fe-804c-903aa1a1c848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:23.555510 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232746f-a5c1-43fe-804c-903aa1a1c848 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:23.555788 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232746f-a5c1-43fe-804c-903aa1a1c848 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:23.556090 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232746f-a5c1-43fe-804c-903aa1a1c848 None admin] PATH_INFO: `/v3/users/039465954f9c46e08badce259ad891c8` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:23.557219 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232746f-a5c1-43fe-804c-903aa1a1c848 None admin] RBAC: Authorizing `identity:get_user(user_id=039465954f9c46e08badce259ad891c8)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:23.558751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232746f-a5c1-43fe-804c-903aa1a1c848 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:23.560356 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 158/328] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:44:23 2022] GET /identity/v3/users/039465954f9c46e08badce259ad891c8 => generated 293 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:23.566811 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232746f-a5c1-43fe-804c-903aa1a1c848 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:23.599517 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 171/329] 10.222.0.16 () {60 vars in 1134 bytes} [Wed Aug 31 10:44:23 2022] POST /identity/v3/groups => generated 226 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:44:23.636118 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643946f-cfb3-4d13-9313-2622b900c472 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:23.682563 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232746f-a5c1-43fe-804c-903aa1a1c848 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:23.692669 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:23.703108 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:23.720958 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f247eca9-bc0e-4cf5-b833-5729a2548239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:23.721579 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247eca9-bc0e-4cf5-b833-5729a2548239 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:23.721855 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247eca9-bc0e-4cf5-b833-5729a2548239 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:23.722123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247eca9-bc0e-4cf5-b833-5729a2548239 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:23.723012 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f247eca9-bc0e-4cf5-b833-5729a2548239 None admin] RBAC: Adding query filter params (role.id=2278da3c52644a819d210b5f9c20b6c1, user.id=039465954f9c46e08badce259ad891c8, scope.system=all) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:23.723320 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f247eca9-bc0e-4cf5-b833-5729a2548239 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:23.724792 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f247eca9-bc0e-4cf5-b833-5729a2548239 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:23.739210 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 159/330] 10.222.0.16 () {56 vars in 1308 bytes} [Wed Aug 31 10:44:23 2022] GET /identity/v3/role_assignments?user.id=039465954f9c46e08badce259ad891c8&scope.system=all&role.id=2278da3c52644a819d210b5f9c20b6c1 => generated 510 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:44:23.759366 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643946f-cfb3-4d13-9313-2622b900c472 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:23.800812 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab584e6c-20eb-49c7-a9c1-07c9bc25e353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:23.801467 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab584e6c-20eb-49c7-a9c1-07c9bc25e353 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:23.803407 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab584e6c-20eb-49c7-a9c1-07c9bc25e353 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:23.803407 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab584e6c-20eb-49c7-a9c1-07c9bc25e353 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:23.817859 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab584e6c-20eb-49c7-a9c1-07c9bc25e353 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:23.823718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-ab584e6c-20eb-49c7-a9c1-07c9bc25e353 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:44:23.823718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:23.823718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:23.823718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:23.823718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:23.823718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:23.823718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:23.823718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:23.823718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:23.823718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:23.823718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:23.823718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:23.823718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:23.823718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:23.823718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:23.823718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:23.824655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:23.824655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:23.824655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:23.824655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:23.824655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:23.824655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:23.824655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:23.824655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:23.824655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:23.824655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:23.824655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:23.824655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:23.824655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:23.824655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:23.824655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:23.824655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:23.825418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:23.825418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:23.825418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:23.825418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:23.825418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:23.825418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:23.825418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:23.825418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:23.825418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:23.825418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:23.825418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:44:23.825418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:23.826788 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 172/331] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:44:23.831744 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f247eca9-bc0e-4cf5-b833-5729a2548239 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:23.948515 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f247eca9-bc0e-4cf5-b833-5729a2548239 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:23.988515 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308931c2-3928-4ea3-9fa4-529dd2c0551e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:23.989125 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308931c2-3928-4ea3-9fa4-529dd2c0551e None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:23.989400 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308931c2-3928-4ea3-9fa4-529dd2c0551e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:23.989667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308931c2-3928-4ea3-9fa4-529dd2c0551e None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:23.990333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308931c2-3928-4ea3-9fa4-529dd2c0551e None admin] RBAC: Adding query filter params (name=reader) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:23.990723 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308931c2-3928-4ea3-9fa4-529dd2c0551e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:23.992086 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308931c2-3928-4ea3-9fa4-529dd2c0551e None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:24.001478 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 160/332] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 31 10:44:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:24.006942 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab584e6c-20eb-49c7-a9c1-07c9bc25e353 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:24.128353 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab584e6c-20eb-49c7-a9c1-07c9bc25e353 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:24.139592 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:24.167679 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618aa833-214c-4c8b-805b-74ed905ce7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:24.168654 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618aa833-214c-4c8b-805b-74ed905ce7c4 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:24.169150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618aa833-214c-4c8b-805b-74ed905ce7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:24.169620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618aa833-214c-4c8b-805b-74ed905ce7c4 None admin] PATH_INFO: `/v3/users/346505f36a3f43749e066c171b35ca66` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:24.170975 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618aa833-214c-4c8b-805b-74ed905ce7c4 None admin] RBAC: Authorizing `identity:get_user(user_id=346505f36a3f43749e066c171b35ca66)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:24.172422 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618aa833-214c-4c8b-805b-74ed905ce7c4 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:24.174322 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 173/333] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:44:24 2022] GET /identity/v3/users/346505f36a3f43749e066c171b35ca66 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:24.180181 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308931c2-3928-4ea3-9fa4-529dd2c0551e None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:24.231884 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:24.304377 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308931c2-3928-4ea3-9fa4-529dd2c0551e None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:24.346893 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eec2f9-7f89-45d3-ad06-321285ab2aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:24.347898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eec2f9-7f89-45d3-ad06-321285ab2aea None admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:24.348443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eec2f9-7f89-45d3-ad06-321285ab2aea None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:24.348908 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eec2f9-7f89-45d3-ad06-321285ab2aea None admin] PATH_INFO: `/v3/system/users/346505f36a3f43749e066c171b35ca66/roles/5585e95a315947d1ad314d7978b46e3a` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:24.351469 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eec2f9-7f89-45d3-ad06-321285ab2aea None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=346505f36a3f43749e066c171b35ca66, role_id=5585e95a315947d1ad314d7978b46e3a)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:24.353236 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eec2f9-7f89-45d3-ad06-321285ab2aea None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:24.441559 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 161/334] 10.222.0.16 () {58 vars in 1334 bytes} [Wed Aug 31 10:44:24 2022] PUT /identity/v3/system/users/346505f36a3f43749e066c171b35ca66/roles/5585e95a315947d1ad314d7978b46e3a => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:44:25.903479 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211df7f-28d2-45ef-9c17-e273dbe3c5c5 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:25.904855 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211df7f-28d2-45ef-9c17-e273dbe3c5c5 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:25.905351 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211df7f-28d2-45ef-9c17-e273dbe3c5c5 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:25.907193 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 174/335] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:25.917189 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058dee0a-55f2-4834-94f5-5cd9bec976e8 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:25.917704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058dee0a-55f2-4834-94f5-5cd9bec976e8 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:25.917704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058dee0a-55f2-4834-94f5-5cd9bec976e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:25.942243 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-058dee0a-55f2-4834-94f5-5cd9bec976e8 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:25.943482 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-058dee0a-55f2-4834-94f5-5cd9bec976e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:26.040767 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 162/336] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:26.362070 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5bf0e-71f6-4da0-b916-1ba17fa511d2 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:26.362761 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5bf0e-71f6-4da0-b916-1ba17fa511d2 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:26.363276 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5bf0e-71f6-4da0-b916-1ba17fa511d2 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:26.365050 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 175/337] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:26.371461 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f471e2-4427-4cd9-9016-1733b8bfcc43 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:26.371748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f471e2-4427-4cd9-9016-1733b8bfcc43 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:26.372027 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f471e2-4427-4cd9-9016-1733b8bfcc43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:26.384198 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5bf0e-71f6-4da0-b916-1ba17fa511d2 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:26.397316 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-27f471e2-4427-4cd9-9016-1733b8bfcc43 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:26.398956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-27f471e2-4427-4cd9-9016-1733b8bfcc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:26.442907 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:26.540083 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 163/338] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:26.574179 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5bf0e-71f6-4da0-b916-1ba17fa511d2 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:26.620737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfebf827-954e-4288-bd38-e001ed40c575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:26.621347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebf827-954e-4288-bd38-e001ed40c575 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:26.621625 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebf827-954e-4288-bd38-e001ed40c575 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:26.621899 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebf827-954e-4288-bd38-e001ed40c575 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:26.640354 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfebf827-954e-4288-bd38-e001ed40c575 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:26.645802 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-dfebf827-954e-4288-bd38-e001ed40c575 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:44:26.645802 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:26.645802 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:26.645802 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:26.645802 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:26.645802 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:26.645802 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:26.645802 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:26.645802 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:26.645802 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:26.645802 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:26.645802 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:26.645802 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:26.645802 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:26.645802 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:26.645802 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:26.646718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:26.646718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:26.646718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:26.646718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:26.646718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:26.646718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:26.646718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:26.646718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:26.646718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:26.646718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:26.646718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:26.646718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:26.646718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:26.646718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:26.646718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:26.646718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:26.647490 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:26.647490 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:26.647490 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:26.647490 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:26.647490 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:26.647490 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:26.647490 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:26.647490 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:26.647490 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:26.647490 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:26.647490 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:44:26.647490 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:26.647490 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 176/339] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:44:26.653012 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f471e2-4427-4cd9-9016-1733b8bfcc43 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:26.770067 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f471e2-4427-4cd9-9016-1733b8bfcc43 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:26.809098 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9885e35-4a20-408c-b3ae-a6f156a5384f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:26.809719 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9885e35-4a20-408c-b3ae-a6f156a5384f None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:26.809998 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9885e35-4a20-408c-b3ae-a6f156a5384f None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:26.810410 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9885e35-4a20-408c-b3ae-a6f156a5384f None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:26.811914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9885e35-4a20-408c-b3ae-a6f156a5384f None admin] RBAC: Adding query filter params (name=member) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:26.812222 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9885e35-4a20-408c-b3ae-a6f156a5384f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:26.813596 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9885e35-4a20-408c-b3ae-a6f156a5384f None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:26.826862 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 164/340] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 31 10:44:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:26.832290 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfebf827-954e-4288-bd38-e001ed40c575 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:26.899398 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:26.982673 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfebf827-954e-4288-bd38-e001ed40c575 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:27.022625 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b0d54-8633-474e-a364-fd042020977a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:27.023242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b0d54-8633-474e-a364-fd042020977a None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:27.023524 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b0d54-8633-474e-a364-fd042020977a None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:27.023794 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b0d54-8633-474e-a364-fd042020977a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:27.043976 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8da797-d27b-4f00-9adc-9d364070f900 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:27.044944 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8da797-d27b-4f00-9adc-9d364070f900 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:27.045445 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8da797-d27b-4f00-9adc-9d364070f900 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:27.047132 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 165/341] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:27.053034 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c89c2-62de-43b4-822a-bd0e4f8d15cb None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:27.053620 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c89c2-62de-43b4-822a-bd0e4f8d15cb None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:27.054095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c89c2-62de-43b4-822a-bd0e4f8d15cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:27.086299 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-093c89c2-62de-43b4-822a-bd0e4f8d15cb None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:27.087899 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-093c89c2-62de-43b4-822a-bd0e4f8d15cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:27.104677 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b0d54-8633-474e-a364-fd042020977a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:27.104677 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b0d54-8633-474e-a364-fd042020977a None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:27.118624 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:27.162846 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:27.200783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 166/342] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:27.216765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c89c2-62de-43b4-822a-bd0e4f8d15cb None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:27.223678 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 177/343] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:44:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:44:27.228220 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b0d54-8633-474e-a364-fd042020977a None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:27.332885 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c89c2-62de-43b4-822a-bd0e4f8d15cb None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:27.374889 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f001ef-5c38-4d97-b311-71064d2b3c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:27.375905 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f001ef-5c38-4d97-b311-71064d2b3c2e None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:27.376388 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f001ef-5c38-4d97-b311-71064d2b3c2e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:27.376860 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f001ef-5c38-4d97-b311-71064d2b3c2e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:27.383072 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b0d54-8633-474e-a364-fd042020977a None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:27.388811 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f001ef-5c38-4d97-b311-71064d2b3c2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:27.399895 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-75f001ef-5c38-4d97-b311-71064d2b3c2e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:44:27.399895 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:27.399895 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:27.399895 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:27.399895 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:27.399895 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:27.399895 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:27.399895 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:27.399895 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:27.399895 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:27.399895 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:27.399895 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:27.399895 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:27.399895 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:27.399895 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:27.399895 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:27.401017 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:27.401017 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:27.401017 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:27.401017 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:27.401017 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:27.401017 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:27.401017 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:27.401017 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:27.401017 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:27.401017 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:27.401017 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:27.401017 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:27.401017 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:27.401017 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:27.401017 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:27.401017 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:27.401972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:27.401972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:27.401972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:27.401972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:27.401972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:27.401972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:27.401972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:27.401972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:27.401972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:27.401972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:27.401972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:44:27.401972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:27.411795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 167/344] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 31 10:44:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:44:27.417894 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f001ef-5c38-4d97-b311-71064d2b3c2e None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:27.453604 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:27.461307 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d6747-a79d-4bfc-8c37-9af47c73fb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:27.470344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d6747-a79d-4bfc-8c37-9af47c73fb39 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:27.470344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d6747-a79d-4bfc-8c37-9af47c73fb39 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:27.470344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d6747-a79d-4bfc-8c37-9af47c73fb39 None admin] PATH_INFO: `/v3/projects/8d08421140694791b18a512ea16981a6` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:27.470344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d6747-a79d-4bfc-8c37-9af47c73fb39 None admin] RBAC: Authorizing `identity:get_project(project_id=8d08421140694791b18a512ea16981a6)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:27.470344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d6747-a79d-4bfc-8c37-9af47c73fb39 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:27.470344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 178/345] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:27 2022] GET /identity/v3/projects/8d08421140694791b18a512ea16981a6 => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:27.474808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d6747-a79d-4bfc-8c37-9af47c73fb39 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:27.517377 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:27.639331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f001ef-5c38-4d97-b311-71064d2b3c2e None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:27.677299 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d6747-a79d-4bfc-8c37-9af47c73fb39 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:27.716372 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35abd799-044b-4fa5-8218-65edbbf63658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:27.718545 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abd799-044b-4fa5-8218-65edbbf63658 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:27.718991 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abd799-044b-4fa5-8218-65edbbf63658 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:27.719515 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abd799-044b-4fa5-8218-65edbbf63658 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:27.720373 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35abd799-044b-4fa5-8218-65edbbf63658 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:27.720850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35abd799-044b-4fa5-8218-65edbbf63658 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:27.722425 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35abd799-044b-4fa5-8218-65edbbf63658 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:27.726086 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd626cc7-0f42-4cda-a35e-bbe7a174e425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:27.726832 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd626cc7-0f42-4cda-a35e-bbe7a174e425 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:27.727183 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd626cc7-0f42-4cda-a35e-bbe7a174e425 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:27.727523 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd626cc7-0f42-4cda-a35e-bbe7a174e425 None admin] PATH_INFO: `/v3/groups/b850e638b5bc42f7a7568111b339d0e1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:27.733565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 168/346] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:44:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:27.734753 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35abd799-044b-4fa5-8218-65edbbf63658 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:27.739358 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd626cc7-0f42-4cda-a35e-bbe7a174e425 None admin] RBAC: Authorizing `identity:get_group(group_id=b850e638b5bc42f7a7568111b339d0e1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:27.741179 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd626cc7-0f42-4cda-a35e-bbe7a174e425 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:27.744317 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 179/347] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 31 10:44:27 2022] GET /identity/v3/groups/b850e638b5bc42f7a7568111b339d0e1 => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:27.745457 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd626cc7-0f42-4cda-a35e-bbe7a174e425 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:27.840258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:27.848316 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:27.874707 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd626cc7-0f42-4cda-a35e-bbe7a174e425 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:27.880545 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35abd799-044b-4fa5-8218-65edbbf63658 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:27.918272 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a921a7-088b-481f-aae6-50b3f17a02c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:27.920119 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a921a7-088b-481f-aae6-50b3f17a02c4 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:27.920598 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a921a7-088b-481f-aae6-50b3f17a02c4 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:27.921113 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a921a7-088b-481f-aae6-50b3f17a02c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:27.925314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89b0cb-4743-4ab8-9637-5382f583482a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:27.926831 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89b0cb-4743-4ab8-9637-5382f583482a None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:27.927293 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89b0cb-4743-4ab8-9637-5382f583482a None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:27.927772 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89b0cb-4743-4ab8-9637-5382f583482a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:27.943602 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a89b0cb-4743-4ab8-9637-5382f583482a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:27.953740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-0a89b0cb-4743-4ab8-9637-5382f583482a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:44:27.953740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:27.953740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:27.953740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:27.953740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:27.953740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:27.953740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:27.953740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:27.953740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:27.953740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:27.953740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:27.953740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:27.953740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:27.953740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:27.953740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:27.953740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:27.954966 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:27.954966 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:27.954966 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:27.954966 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:27.954966 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:27.954966 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:27.954966 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:27.954966 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:27.954966 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:27.954966 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:27.954966 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:27.954966 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:27.954966 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:27.954966 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:27.954966 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:27.954966 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:27.955960 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:27.955960 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:27.955960 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:27.955960 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:27.955960 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:27.955960 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:27.955960 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:27.955960 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:27.955960 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:27.955960 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:27.955960 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:44:27.955960 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:27.958234 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 169/348] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 31 10:44:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:27.959831 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89b0cb-4743-4ab8-9637-5382f583482a None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:28.013128 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a921a7-088b-481f-aae6-50b3f17a02c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:28.014739 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a921a7-088b-481f-aae6-50b3f17a02c4 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:28.082469 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89b0cb-4743-4ab8-9637-5382f583482a None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:28.106629 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:28.121051 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d95093-5a8d-4f6c-ae26-aa64a0a80021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:28.127833 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d95093-5a8d-4f6c-ae26-aa64a0a80021 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:28.127833 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d95093-5a8d-4f6c-ae26-aa64a0a80021 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:28.127833 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d95093-5a8d-4f6c-ae26-aa64a0a80021 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:28.127833 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d95093-5a8d-4f6c-ae26-aa64a0a80021 None admin] RBAC: Adding query filter params (group.id=b850e638b5bc42f7a7568111b339d0e1, role.id=2278da3c52644a819d210b5f9c20b6c1, scope.project.id=8d08421140694791b18a512ea16981a6) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:28.127833 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d95093-5a8d-4f6c-ae26-aa64a0a80021 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:28.127833 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d95093-5a8d-4f6c-ae26-aa64a0a80021 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:28.129560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:28.132526 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 170/349] 10.222.0.16 () {56 vars in 1376 bytes} [Wed Aug 31 10:44:27 2022] GET /identity/v3/role_assignments?group.id=b850e638b5bc42f7a7568111b339d0e1&scope.project.id=8d08421140694791b18a512ea16981a6&role.id=2278da3c52644a819d210b5f9c20b6c1 => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:28.135085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fe47e-3f95-4fad-92f2-5ab7faac28a1 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:28.135372 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fe47e-3f95-4fad-92f2-5ab7faac28a1 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:28.135649 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fe47e-3f95-4fad-92f2-5ab7faac28a1 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:28.137099 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 171/350] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:28.142180 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ae081-24e2-4c44-9411-ec82076212e3 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:28.144852 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ae081-24e2-4c44-9411-ec82076212e3 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:28.144852 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ae081-24e2-4c44-9411-ec82076212e3 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:28.144852 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 172/351] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:28.145270 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ae081-24e2-4c44-9411-ec82076212e3 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:28.151790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 180/352] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:44:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:44:28.153113 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc1a72-15bb-4d82-a07f-00490d595850 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:28.153400 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc1a72-15bb-4d82-a07f-00490d595850 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:28.153671 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc1a72-15bb-4d82-a07f-00490d595850 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:28.155343 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 181/353] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:28.161081 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec7101-04c1-4d89-957c-4cf403e40d1c None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:28.161613 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec7101-04c1-4d89-957c-4cf403e40d1c None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:28.161889 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec7101-04c1-4d89-957c-4cf403e40d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:28.185010 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-71ec7101-04c1-4d89-957c-4cf403e40d1c None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:28.186892 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-71ec7101-04c1-4d89-957c-4cf403e40d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:28.295726 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 182/354] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:28.297967 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9af5c-e2b3-4777-bd83-bf2648716fef None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:28.297967 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9af5c-e2b3-4777-bd83-bf2648716fef None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:28.297967 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9af5c-e2b3-4777-bd83-bf2648716fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:28.303288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ae081-24e2-4c44-9411-ec82076212e3 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:28.318494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-38e9af5c-e2b3-4777-bd83-bf2648716fef None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:28.319890 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-38e9af5c-e2b3-4777-bd83-bf2648716fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:28.345515 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5f0f9-c24e-41fd-b5d7-0a8043353d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:28.346137 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5f0f9-c24e-41fd-b5d7-0a8043353d00 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:28.346456 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5f0f9-c24e-41fd-b5d7-0a8043353d00 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:28.346740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5f0f9-c24e-41fd-b5d7-0a8043353d00 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:28.347410 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f5f0f9-c24e-41fd-b5d7-0a8043353d00 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:28.347716 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f5f0f9-c24e-41fd-b5d7-0a8043353d00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:28.349083 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f5f0f9-c24e-41fd-b5d7-0a8043353d00 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:28.358588 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 173/355] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:44:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:28.359341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5f0f9-c24e-41fd-b5d7-0a8043353d00 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:28.400082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 183/356] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:28.401641 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f5d23-5da6-4426-8305-d8a6a4ab2ce0 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:28.402105 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f5d23-5da6-4426-8305-d8a6a4ab2ce0 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:28.402574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f5d23-5da6-4426-8305-d8a6a4ab2ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:28.423093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-7b0f5d23-5da6-4426-8305-d8a6a4ab2ce0 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:28.424724 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-7b0f5d23-5da6-4426-8305-d8a6a4ab2ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:28.433589 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:28.456728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:28.474821 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5f0f9-c24e-41fd-b5d7-0a8043353d00 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:28.517791 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf27d3-12fa-4dda-9845-13b128112b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:28.519083 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf27d3-12fa-4dda-9845-13b128112b19 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:28.519527 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf27d3-12fa-4dda-9845-13b128112b19 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:28.519941 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf27d3-12fa-4dda-9845-13b128112b19 None admin] PATH_INFO: `/v3/projects/8d08421140694791b18a512ea16981a6` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:28.521633 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf27d3-12fa-4dda-9845-13b128112b19 None admin] RBAC: Authorizing `identity:get_project(project_id=8d08421140694791b18a512ea16981a6)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:28.523666 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf27d3-12fa-4dda-9845-13b128112b19 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:28.525851 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 174/357] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:28 2022] GET /identity/v3/projects/8d08421140694791b18a512ea16981a6 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:28.527035 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf27d3-12fa-4dda-9845-13b128112b19 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:28.541164 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 184/358] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:28.542435 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f5d23-5da6-4426-8305-d8a6a4ab2ce0 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:28.643969 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf27d3-12fa-4dda-9845-13b128112b19 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:28.680366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:28.682822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cccf54-436a-437d-a7d2-bd2e85686780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:28.683754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cccf54-436a-437d-a7d2-bd2e85686780 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:28.684313 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cccf54-436a-437d-a7d2-bd2e85686780 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:28.684735 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cccf54-436a-437d-a7d2-bd2e85686780 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:28.763040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cccf54-436a-437d-a7d2-bd2e85686780 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:28.765806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cccf54-436a-437d-a7d2-bd2e85686780 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:28.792097 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f5d23-5da6-4426-8305-d8a6a4ab2ce0 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:28.842968 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d82bd-5352-4c0f-b4e3-2d7e1efedb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:28.843893 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d82bd-5352-4c0f-b4e3-2d7e1efedb91 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:28.844172 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d82bd-5352-4c0f-b4e3-2d7e1efedb91 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:28.844446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d82bd-5352-4c0f-b4e3-2d7e1efedb91 None admin] PATH_INFO: `/v3/groups/b850e638b5bc42f7a7568111b339d0e1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:28.846076 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d82bd-5352-4c0f-b4e3-2d7e1efedb91 None admin] RBAC: Authorizing `identity:get_group(group_id=b850e638b5bc42f7a7568111b339d0e1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:28.847983 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d82bd-5352-4c0f-b4e3-2d7e1efedb91 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:28.854709 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 185/359] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 31 10:44:28 2022] GET /identity/v3/groups/b850e638b5bc42f7a7568111b339d0e1 => generated 226 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:28.855585 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d82bd-5352-4c0f-b4e3-2d7e1efedb91 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:28.891920 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:28.906157 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 175/360] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:44:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:44:28.907119 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cccf54-436a-437d-a7d2-bd2e85686780 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:28.948619 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:28.989811 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d82bd-5352-4c0f-b4e3-2d7e1efedb91 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:29.038456 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653aba7-7ff1-444c-9a0b-bb8f6ed113ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:29.039113 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653aba7-7ff1-444c-9a0b-bb8f6ed113ef None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:29.039438 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653aba7-7ff1-444c-9a0b-bb8f6ed113ef None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:29.039711 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653aba7-7ff1-444c-9a0b-bb8f6ed113ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:29.052407 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cccf54-436a-437d-a7d2-bd2e85686780 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:29.057655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c653aba7-7ff1-444c-9a0b-bb8f6ed113ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:29.064681 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-c653aba7-7ff1-444c-9a0b-bb8f6ed113ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:44:29.064681 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:29.064681 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:29.064681 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:29.064681 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:29.064681 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:29.064681 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:29.064681 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:29.064681 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:29.064681 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:29.064681 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:29.064681 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:29.064681 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:29.064681 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:29.064681 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:29.064681 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:29.065554 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:29.065554 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:29.065554 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:29.065554 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:29.065554 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:29.065554 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:29.065554 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:29.065554 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:29.065554 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:29.065554 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:29.065554 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:29.065554 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:29.065554 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:29.065554 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:29.065554 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:29.065554 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:29.066340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:29.066340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:29.066340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:29.066340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:29.066340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:29.066340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:29.066340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:29.066340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:29.066340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:29.066340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:29.066340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:44:29.066340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:29.066340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 186/361] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:44:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:29.067492 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653aba7-7ff1-444c-9a0b-bb8f6ed113ef None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:29.098171 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4caefef-a739-42d8-a8b9-4a2ef77a0f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:29.098894 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4caefef-a739-42d8-a8b9-4a2ef77a0f4e None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:29.099279 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4caefef-a739-42d8-a8b9-4a2ef77a0f4e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:29.100115 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4caefef-a739-42d8-a8b9-4a2ef77a0f4e None admin] PATH_INFO: `/v3/roles/member` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:29.120672 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4caefef-a739-42d8-a8b9-4a2ef77a0f4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:29.126082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-d4caefef-a739-42d8-a8b9-4a2ef77a0f4e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:44:29.126082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:29.126082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:29.126082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:29.126082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:29.126082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:29.126082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:29.126082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:29.126082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:29.126082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:29.126082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:29.126082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:29.126082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:29.126082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:29.126082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:29.126082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:29.127070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:29.127070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:29.127070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:29.127070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:29.127070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:29.127070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:29.127070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:29.127070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:29.127070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:29.127070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:29.127070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:29.127070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:29.127070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:29.127070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:29.127070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:29.127070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:29.127858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:29.127858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:29.127858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:29.127858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:29.127858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:29.127858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:29.127858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:29.127858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:29.127858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:29.127858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:29.127858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:44:29.127858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:29.129880 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 176/362] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:29.130753 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4caefef-a739-42d8-a8b9-4a2ef77a0f4e None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:29.206305 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:29.227540 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653aba7-7ff1-444c-9a0b-bb8f6ed113ef None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:29.246578 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:29.268461 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4caefef-a739-42d8-a8b9-4a2ef77a0f4e None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:29.281672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f16ac-7e87-45d9-be5a-9b418de5743a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:29.282796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f16ac-7e87-45d9-be5a-9b418de5743a None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:29.283236 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f16ac-7e87-45d9-be5a-9b418de5743a None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:29.283697 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f16ac-7e87-45d9-be5a-9b418de5743a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:29.284613 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f16ac-7e87-45d9-be5a-9b418de5743a None admin] RBAC: Adding query filter params (group.id=b850e638b5bc42f7a7568111b339d0e1, role.id=31d4b1dc082c41f7b6f18c338e7b51df, scope.project.id=8d08421140694791b18a512ea16981a6) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:29.285158 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f16ac-7e87-45d9-be5a-9b418de5743a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:29.287121 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f16ac-7e87-45d9-be5a-9b418de5743a None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:29.298529 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 187/363] 10.222.0.16 () {56 vars in 1376 bytes} [Wed Aug 31 10:44:29 2022] GET /identity/v3/role_assignments?group.id=b850e638b5bc42f7a7568111b339d0e1&scope.project.id=8d08421140694791b18a512ea16981a6&role.id=31d4b1dc082c41f7b6f18c338e7b51df => generated 260 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:29.299525 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f16ac-7e87-45d9-be5a-9b418de5743a None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:29.311905 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f69ec-eb52-4aed-8b09-b38b80d9bc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:29.312568 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f69ec-eb52-4aed-8b09-b38b80d9bc2f None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:29.312845 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f69ec-eb52-4aed-8b09-b38b80d9bc2f None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:29.313118 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f69ec-eb52-4aed-8b09-b38b80d9bc2f None admin] PATH_INFO: `/v3/projects/08b4540d439e44adb9398aec4130989a` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:29.314361 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f69ec-eb52-4aed-8b09-b38b80d9bc2f None admin] RBAC: Authorizing `identity:get_project(project_id=08b4540d439e44adb9398aec4130989a)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:29.315898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f69ec-eb52-4aed-8b09-b38b80d9bc2f None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:29.317971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 177/364] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:29 2022] GET /identity/v3/projects/08b4540d439e44adb9398aec4130989a => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:29.318846 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f69ec-eb52-4aed-8b09-b38b80d9bc2f None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:29.427109 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f16ac-7e87-45d9-be5a-9b418de5743a None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:29.470301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ef695-c622-4a39-a716-6d8b05d40339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:29.471335 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ef695-c622-4a39-a716-6d8b05d40339 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:29.471778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ef695-c622-4a39-a716-6d8b05d40339 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:29.472216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ef695-c622-4a39-a716-6d8b05d40339 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:29.479815 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f69ec-eb52-4aed-8b09-b38b80d9bc2f None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:29.488898 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ef695-c622-4a39-a716-6d8b05d40339 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:29.495671 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-2c7ef695-c622-4a39-a716-6d8b05d40339 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:44:29.495671 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:29.495671 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:29.495671 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:29.495671 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:29.495671 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:29.495671 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:29.495671 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:29.495671 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:29.495671 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:29.495671 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:29.495671 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:29.495671 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:29.495671 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:29.495671 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:29.495671 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:29.496583 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:29.496583 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:29.496583 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:29.496583 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:29.496583 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:29.496583 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:29.496583 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:29.496583 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:29.496583 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:29.496583 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:29.496583 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:29.496583 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:29.496583 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:29.496583 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:29.496583 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:29.496583 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:29.497383 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:29.497383 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:29.497383 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:29.497383 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:29.497383 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:29.497383 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:29.497383 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:29.497383 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:29.497383 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:29.497383 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:29.497383 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:44:29.497383 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:29.498753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 188/365] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:29.499299 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ef695-c622-4a39-a716-6d8b05d40339 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:29.520313 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:29.539948 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70548c90-b6e0-40a2-a007-480d026bba8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:29.540594 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70548c90-b6e0-40a2-a007-480d026bba8e None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:29.540871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70548c90-b6e0-40a2-a007-480d026bba8e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:29.541139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70548c90-b6e0-40a2-a007-480d026bba8e None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:29.541816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70548c90-b6e0-40a2-a007-480d026bba8e None admin] RBAC: Adding query filter params (name=admin) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:29.542114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70548c90-b6e0-40a2-a007-480d026bba8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:29.543723 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70548c90-b6e0-40a2-a007-480d026bba8e None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:29.556162 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 178/366] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:29.558301 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70548c90-b6e0-40a2-a007-480d026bba8e None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:29.598975 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:29.663506 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ef695-c622-4a39-a716-6d8b05d40339 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:29.713675 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c1d17-491d-4ab4-882f-c6600e1a445b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:29.714470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c1d17-491d-4ab4-882f-c6600e1a445b None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:29.714757 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c1d17-491d-4ab4-882f-c6600e1a445b None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:29.715027 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c1d17-491d-4ab4-882f-c6600e1a445b None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:29.715705 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c1d17-491d-4ab4-882f-c6600e1a445b None admin] RBAC: Adding query filter params (name=member) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:29.716010 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c1d17-491d-4ab4-882f-c6600e1a445b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:29.718503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c1d17-491d-4ab4-882f-c6600e1a445b None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:29.727856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70548c90-b6e0-40a2-a007-480d026bba8e None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:29.734418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 189/367] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 31 10:44:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:29.734418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c1d17-491d-4ab4-882f-c6600e1a445b None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:29.785870 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b649340-43c5-4a2e-97ec-f39b9e0669be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:29.786976 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b649340-43c5-4a2e-97ec-f39b9e0669be None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:29.787261 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b649340-43c5-4a2e-97ec-f39b9e0669be None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:29.787533 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b649340-43c5-4a2e-97ec-f39b9e0669be None admin] PATH_INFO: `/v3/groups/b850e638b5bc42f7a7568111b339d0e1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:29.788932 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b649340-43c5-4a2e-97ec-f39b9e0669be None admin] RBAC: Authorizing `identity:get_group(group_id=b850e638b5bc42f7a7568111b339d0e1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:29.790681 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b649340-43c5-4a2e-97ec-f39b9e0669be None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:29.793277 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 179/368] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 31 10:44:29 2022] GET /identity/v3/groups/b850e638b5bc42f7a7568111b339d0e1 => generated 226 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:29.793852 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b649340-43c5-4a2e-97ec-f39b9e0669be None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:29.827808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:29.861214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c1d17-491d-4ab4-882f-c6600e1a445b None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:29.901847 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71893107-769b-4eb4-9c29-99759bf5d26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:29.902598 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71893107-769b-4eb4-9c29-99759bf5d26e None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:29.902874 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71893107-769b-4eb4-9c29-99759bf5d26e None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:29.903142 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71893107-769b-4eb4-9c29-99759bf5d26e None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:29.903816 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71893107-769b-4eb4-9c29-99759bf5d26e None admin] RBAC: Adding query filter params (name=reader) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:29.904309 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71893107-769b-4eb4-9c29-99759bf5d26e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:29.906358 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71893107-769b-4eb4-9c29-99759bf5d26e None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:29.910570 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:29.917471 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 190/369] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 31 10:44:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:29.924241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71893107-769b-4eb4-9c29-99759bf5d26e None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:29.924372 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b649340-43c5-4a2e-97ec-f39b9e0669be None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:29.986534 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f54e2f-0298-4f8d-b0d6-31edaf3c5005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:29.987462 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f54e2f-0298-4f8d-b0d6-31edaf3c5005 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:29.988404 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f54e2f-0298-4f8d-b0d6-31edaf3c5005 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:29.988841 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f54e2f-0298-4f8d-b0d6-31edaf3c5005 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:30.049395 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71893107-769b-4eb4-9c29-99759bf5d26e None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:30.088395 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f54e2f-0298-4f8d-b0d6-31edaf3c5005 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:30.089784 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f54e2f-0298-4f8d-b0d6-31edaf3c5005 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:30.100363 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d908e9-9ed6-4160-9d46-5bee7952fe03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:30.101325 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d908e9-9ed6-4160-9d46-5bee7952fe03 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:30.101803 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d908e9-9ed6-4160-9d46-5bee7952fe03 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:30.102797 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d908e9-9ed6-4160-9d46-5bee7952fe03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:30.114442 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:30.224044 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d908e9-9ed6-4160-9d46-5bee7952fe03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:30.226750 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d908e9-9ed6-4160-9d46-5bee7952fe03 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:30.229509 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:30.248681 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 180/370] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:44:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:44:30.249403 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f54e2f-0298-4f8d-b0d6-31edaf3c5005 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:30.361900 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 191/371] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:44:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:44:30.363214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d908e9-9ed6-4160-9d46-5bee7952fe03 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:30.376621 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f54e2f-0298-4f8d-b0d6-31edaf3c5005 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:30.406736 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:30.420711 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3690225-7f74-407b-adbc-2424b2c66e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:30.421450 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3690225-7f74-407b-adbc-2424b2c66e4a None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:30.421795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3690225-7f74-407b-adbc-2424b2c66e4a None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:30.422138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3690225-7f74-407b-adbc-2424b2c66e4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:30.425046 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3690225-7f74-407b-adbc-2424b2c66e4a None admin] RBAC: Adding query filter params (group.id=b850e638b5bc42f7a7568111b339d0e1, role.id=31d4b1dc082c41f7b6f18c338e7b51df, scope.project.id=08b4540d439e44adb9398aec4130989a) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:30.425443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3690225-7f74-407b-adbc-2424b2c66e4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:30.427028 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3690225-7f74-407b-adbc-2424b2c66e4a None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:30.436046 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 181/372] 10.222.0.16 () {56 vars in 1376 bytes} [Wed Aug 31 10:44:30 2022] GET /identity/v3/role_assignments?group.id=b850e638b5bc42f7a7568111b339d0e1&scope.project.id=08b4540d439e44adb9398aec4130989a&role.id=31d4b1dc082c41f7b6f18c338e7b51df => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:30.437330 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3690225-7f74-407b-adbc-2424b2c66e4a None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:30.535597 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d908e9-9ed6-4160-9d46-5bee7952fe03 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:30.558825 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:30.614255 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d95d0-8c01-42b3-a5d7-a962b4e5d80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:30.614849 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d95d0-8c01-42b3-a5d7-a962b4e5d80e None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:30.615128 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d95d0-8c01-42b3-a5d7-a962b4e5d80e None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:30.615403 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d95d0-8c01-42b3-a5d7-a962b4e5d80e None admin] PATH_INFO: `/v3/users/346505f36a3f43749e066c171b35ca66` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:30.617422 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d95d0-8c01-42b3-a5d7-a962b4e5d80e None admin] RBAC: Authorizing `identity:get_user(user_id=346505f36a3f43749e066c171b35ca66)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:30.618927 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d95d0-8c01-42b3-a5d7-a962b4e5d80e None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:30.624582 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 192/373] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:44:30 2022] GET /identity/v3/users/346505f36a3f43749e066c171b35ca66 => generated 293 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:30.625464 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d95d0-8c01-42b3-a5d7-a962b4e5d80e None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:30.650528 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3690225-7f74-407b-adbc-2424b2c66e4a None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:30.703884 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac38dac-8bca-4c6c-a859-ec9c9d552a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:30.704880 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac38dac-8bca-4c6c-a859-ec9c9d552a0c None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:30.705334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac38dac-8bca-4c6c-a859-ec9c9d552a0c None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:30.705754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac38dac-8bca-4c6c-a859-ec9c9d552a0c None admin] PATH_INFO: `/v3/projects/776925ef060044f49d8dab7d8928f73c` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:30.711140 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac38dac-8bca-4c6c-a859-ec9c9d552a0c None admin] RBAC: Authorizing `identity:get_project(project_id=776925ef060044f49d8dab7d8928f73c)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:30.712616 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac38dac-8bca-4c6c-a859-ec9c9d552a0c None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:30.714509 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 182/374] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:30 2022] GET /identity/v3/projects/776925ef060044f49d8dab7d8928f73c => generated 344 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:30.718129 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac38dac-8bca-4c6c-a859-ec9c9d552a0c None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:30.832540 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d95d0-8c01-42b3-a5d7-a962b4e5d80e None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:30.839456 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:30.844367 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac38dac-8bca-4c6c-a859-ec9c9d552a0c None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:30.876311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:30.891346 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17df5a5-6fd1-4676-8111-a22c380b0cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:30.898410 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17df5a5-6fd1-4676-8111-a22c380b0cb7 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:30.898410 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17df5a5-6fd1-4676-8111-a22c380b0cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:30.898410 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17df5a5-6fd1-4676-8111-a22c380b0cb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:30.898410 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17df5a5-6fd1-4676-8111-a22c380b0cb7 None admin] RBAC: Adding query filter params (role.id=5585e95a315947d1ad314d7978b46e3a, user.id=346505f36a3f43749e066c171b35ca66, scope.system=all) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:30.898410 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17df5a5-6fd1-4676-8111-a22c380b0cb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:30.898410 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17df5a5-6fd1-4676-8111-a22c380b0cb7 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:30.899875 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ac329-292b-4410-a168-ae585692ebc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:30.900512 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ac329-292b-4410-a168-ae585692ebc9 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:30.900793 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ac329-292b-4410-a168-ae585692ebc9 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:30.901064 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ac329-292b-4410-a168-ae585692ebc9 None admin] PATH_INFO: `/v3/projects/08b4540d439e44adb9398aec4130989a` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:30.903067 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ac329-292b-4410-a168-ae585692ebc9 None admin] RBAC: Authorizing `identity:get_project(project_id=08b4540d439e44adb9398aec4130989a)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:30.905150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ac329-292b-4410-a168-ae585692ebc9 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:30.907812 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 193/375] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:30 2022] GET /identity/v3/projects/08b4540d439e44adb9398aec4130989a => generated 302 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:30.908598 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ac329-292b-4410-a168-ae585692ebc9 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:30.928429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 183/376] 10.222.0.16 () {56 vars in 1308 bytes} [Wed Aug 31 10:44:30 2022] GET /identity/v3/role_assignments?user.id=346505f36a3f43749e066c171b35ca66&scope.system=all&role.id=5585e95a315947d1ad314d7978b46e3a => generated 510 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:30.929500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17df5a5-6fd1-4676-8111-a22c380b0cb7 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:31.093011 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ac329-292b-4410-a168-ae585692ebc9 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:31.156620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4e3f0-9876-45e0-9561-40a57eacc01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:31.157615 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4e3f0-9876-45e0-9561-40a57eacc01a None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:31.158100 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4e3f0-9876-45e0-9561-40a57eacc01a None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:31.158599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4e3f0-9876-45e0-9561-40a57eacc01a None admin] PATH_INFO: `/v3/groups/13cb05d6493d46898a644db37bd7a699` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:31.159720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17df5a5-6fd1-4676-8111-a22c380b0cb7 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:31.167525 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df4e3f0-9876-45e0-9561-40a57eacc01a None admin] RBAC: Authorizing `identity:get_group(group_id=13cb05d6493d46898a644db37bd7a699)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:31.169322 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df4e3f0-9876-45e0-9561-40a57eacc01a None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:31.171665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 194/377] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 31 10:44:30 2022] GET /identity/v3/groups/13cb05d6493d46898a644db37bd7a699 => generated 229 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:31.173094 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b96ddb-70f0-45fb-ba2a-edced6c817cc None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:31.173582 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b96ddb-70f0-45fb-ba2a-edced6c817cc None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:31.174065 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b96ddb-70f0-45fb-ba2a-edced6c817cc None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:31.176786 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 195/378] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:31.183228 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07d466-c83a-4763-871a-6a2177ea4061 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:31.183880 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07d466-c83a-4763-871a-6a2177ea4061 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:31.184345 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07d466-c83a-4763-871a-6a2177ea4061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:31.209689 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-de07d466-c83a-4763-871a-6a2177ea4061 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:31.211447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-de07d466-c83a-4763-871a-6a2177ea4061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:31.212896 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce003bd8-c62e-4d5d-9d8e-f304ae371eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:31.213602 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce003bd8-c62e-4d5d-9d8e-f304ae371eec None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:31.213880 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce003bd8-c62e-4d5d-9d8e-f304ae371eec None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:31.214172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce003bd8-c62e-4d5d-9d8e-f304ae371eec None admin] PATH_INFO: `/v3/groups/b850e638b5bc42f7a7568111b339d0e1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:31.215642 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce003bd8-c62e-4d5d-9d8e-f304ae371eec None admin] RBAC: Authorizing `identity:get_group(group_id=b850e638b5bc42f7a7568111b339d0e1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:31.218934 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce003bd8-c62e-4d5d-9d8e-f304ae371eec None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:31.224724 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 184/379] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 31 10:44:30 2022] GET /identity/v3/groups/b850e638b5bc42f7a7568111b339d0e1 => generated 226 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:31.225852 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce003bd8-c62e-4d5d-9d8e-f304ae371eec None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:31.258429 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:31.284089 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:31.307109 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 196/380] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:31.309378 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de07d466-c83a-4763-871a-6a2177ea4061 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:31.342186 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce003bd8-c62e-4d5d-9d8e-f304ae371eec None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:31.381098 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29a0ed-8067-4735-8d03-ab241b08b9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:31.382086 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29a0ed-8067-4735-8d03-ab241b08b9f3 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:31.382649 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29a0ed-8067-4735-8d03-ab241b08b9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:31.383117 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29a0ed-8067-4735-8d03-ab241b08b9f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:31.384090 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29a0ed-8067-4735-8d03-ab241b08b9f3 None admin] RBAC: Adding query filter params (group.id=13cb05d6493d46898a644db37bd7a699, role.id=13ca33d694c645d6920930d39dc52a0d, scope.project.id=776925ef060044f49d8dab7d8928f73c) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:31.384588 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29a0ed-8067-4735-8d03-ab241b08b9f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:31.386196 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29a0ed-8067-4735-8d03-ab241b08b9f3 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:31.393010 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 185/381] 10.222.0.16 () {56 vars in 1376 bytes} [Wed Aug 31 10:44:31 2022] GET /identity/v3/role_assignments?group.id=13cb05d6493d46898a644db37bd7a699&scope.project.id=776925ef060044f49d8dab7d8928f73c&role.id=13ca33d694c645d6920930d39dc52a0d => generated 260 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:31.431703 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de07d466-c83a-4763-871a-6a2177ea4061 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:31.484260 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f912a-ccf0-47ce-ad25-0d3747799742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:31.485276 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f912a-ccf0-47ce-ad25-0d3747799742 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:31.485753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f912a-ccf0-47ce-ad25-0d3747799742 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:31.486250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f912a-ccf0-47ce-ad25-0d3747799742 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:31.487220 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f912a-ccf0-47ce-ad25-0d3747799742 None admin] RBAC: Adding query filter params (group.id=b850e638b5bc42f7a7568111b339d0e1, role.id=2278da3c52644a819d210b5f9c20b6c1, scope.project.id=08b4540d439e44adb9398aec4130989a) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:31.487746 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f912a-ccf0-47ce-ad25-0d3747799742 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:31.489434 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f912a-ccf0-47ce-ad25-0d3747799742 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:31.500520 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 197/382] 10.222.0.16 () {56 vars in 1376 bytes} [Wed Aug 31 10:44:31 2022] GET /identity/v3/role_assignments?group.id=b850e638b5bc42f7a7568111b339d0e1&scope.project.id=08b4540d439e44adb9398aec4130989a&role.id=2278da3c52644a819d210b5f9c20b6c1 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:31.711507 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553fe41-eb88-47c0-a6f2-7aa45c2fa488 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:31.712115 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553fe41-eb88-47c0-a6f2-7aa45c2fa488 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:31.712584 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553fe41-eb88-47c0-a6f2-7aa45c2fa488 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:31.714510 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 186/383] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:31.720016 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9520a6-5a9e-4b18-a98e-0bb86053c4a9 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:31.720581 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9520a6-5a9e-4b18-a98e-0bb86053c4a9 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:31.721049 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9520a6-5a9e-4b18-a98e-0bb86053c4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:31.740511 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-2c9520a6-5a9e-4b18-a98e-0bb86053c4a9 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:31.742047 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2c9520a6-5a9e-4b18-a98e-0bb86053c4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:31.773462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:31.798499 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553fe41-eb88-47c0-a6f2-7aa45c2fa488 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:31.827693 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 198/384] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:31.970676 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553fe41-eb88-47c0-a6f2-7aa45c2fa488 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:31.990350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:32.018176 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe53a13-725e-47d7-925f-b25cf9f6d419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:32.018867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe53a13-725e-47d7-925f-b25cf9f6d419 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:32.019143 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe53a13-725e-47d7-925f-b25cf9f6d419 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:32.019412 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe53a13-725e-47d7-925f-b25cf9f6d419 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:32.037326 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe53a13-725e-47d7-925f-b25cf9f6d419 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:32.042698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-efe53a13-725e-47d7-925f-b25cf9f6d419 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:44:32.042698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:32.042698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:32.042698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:32.042698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:32.042698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:32.042698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:32.042698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:32.042698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:32.042698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:32.042698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:32.042698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:32.042698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:32.042698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:32.042698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:32.042698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:32.043581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:32.043581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:32.043581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:32.043581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:32.043581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:32.043581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:32.043581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:32.043581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:32.043581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:32.043581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:32.043581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:32.043581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:32.043581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:32.043581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:32.043581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:32.043581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:32.044348 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:32.044348 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:32.044348 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:32.044348 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:32.044348 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:32.044348 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:32.044348 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:32.044348 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:32.044348 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:32.044348 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:32.044348 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:44:32.044348 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:32.046239 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 187/385] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:32.050299 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9520a6-5a9e-4b18-a98e-0bb86053c4a9 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:32.188209 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9520a6-5a9e-4b18-a98e-0bb86053c4a9 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:32.220801 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe53a13-725e-47d7-925f-b25cf9f6d419 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:32.229023 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5961218-3578-4d1b-898b-f15a6aff2dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:32.229653 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5961218-3578-4d1b-898b-f15a6aff2dc0 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:32.229938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5961218-3578-4d1b-898b-f15a6aff2dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:32.230213 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5961218-3578-4d1b-898b-f15a6aff2dc0 None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:32.230941 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5961218-3578-4d1b-898b-f15a6aff2dc0 None admin] RBAC: Adding query filter params (name=member) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:32.231246 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5961218-3578-4d1b-898b-f15a6aff2dc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:32.232726 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5961218-3578-4d1b-898b-f15a6aff2dc0 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:32.246316 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 199/386] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 31 10:44:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:32.249461 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5961218-3578-4d1b-898b-f15a6aff2dc0 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:32.284986 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:32.392145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe53a13-725e-47d7-925f-b25cf9f6d419 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:32.425408 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5961218-3578-4d1b-898b-f15a6aff2dc0 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:32.439265 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5f5e0-f0af-4b4c-b48d-fb3437668413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:32.440022 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5f5e0-f0af-4b4c-b48d-fb3437668413 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:32.440296 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5f5e0-f0af-4b4c-b48d-fb3437668413 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:32.440559 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5f5e0-f0af-4b4c-b48d-fb3437668413 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:32.464880 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac5f5e0-f0af-4b4c-b48d-fb3437668413 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:32.471916 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-2ac5f5e0-f0af-4b4c-b48d-fb3437668413 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:44:32.471916 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:32.471916 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:32.471916 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:32.471916 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:32.471916 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:32.471916 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:32.471916 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:32.471916 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:32.471916 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:32.471916 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:32.471916 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:32.471916 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:32.471916 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:32.471916 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:32.471916 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:32.472778 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:32.472778 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:32.472778 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:32.472778 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:32.472778 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:32.472778 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:32.472778 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:32.472778 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:32.472778 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:32.472778 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:32.472778 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:32.472778 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:32.472778 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:32.472778 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:32.472778 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:32.472778 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:32.473536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:32.473536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:32.473536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:32.473536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:32.473536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:32.473536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:32.473536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:32.473536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:32.473536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:32.473536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:32.473536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:44:32.473536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:32.473536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 188/387] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 31 10:44:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:44:32.480097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5f5e0-f0af-4b4c-b48d-fb3437668413 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:32.516177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea99552e-8ab1-4274-a10f-1b95f7bec2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:32.516828 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99552e-8ab1-4274-a10f-1b95f7bec2f8 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:32.517106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99552e-8ab1-4274-a10f-1b95f7bec2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:32.517379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99552e-8ab1-4274-a10f-1b95f7bec2f8 None admin] PATH_INFO: `/v3/groups/b850e638b5bc42f7a7568111b339d0e1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:32.521415 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea99552e-8ab1-4274-a10f-1b95f7bec2f8 None admin] RBAC: Authorizing `identity:get_group(group_id=b850e638b5bc42f7a7568111b339d0e1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:32.522930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea99552e-8ab1-4274-a10f-1b95f7bec2f8 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:32.532577 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 200/388] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 31 10:44:32 2022] GET /identity/v3/groups/b850e638b5bc42f7a7568111b339d0e1 => generated 226 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:32.533507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea99552e-8ab1-4274-a10f-1b95f7bec2f8 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:32.554316 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:32.689859 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:32.704481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea99552e-8ab1-4274-a10f-1b95f7bec2f8 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:32.736777 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5f5e0-f0af-4b4c-b48d-fb3437668413 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:32.757329 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3000be-9c45-41d6-82b7-5232274e4425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:32.758013 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3000be-9c45-41d6-82b7-5232274e4425 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:32.758291 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3000be-9c45-41d6-82b7-5232274e4425 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:32.758578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3000be-9c45-41d6-82b7-5232274e4425 None admin] PATH_INFO: `/v3/projects/8d08421140694791b18a512ea16981a6` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:32.761784 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3000be-9c45-41d6-82b7-5232274e4425 None admin] RBAC: Authorizing `identity:get_project(project_id=8d08421140694791b18a512ea16981a6)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:32.763853 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3000be-9c45-41d6-82b7-5232274e4425 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:32.765879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 201/389] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:32 2022] GET /identity/v3/projects/8d08421140694791b18a512ea16981a6 => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:32.773933 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3000be-9c45-41d6-82b7-5232274e4425 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:32.808277 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc743b-b2a9-4cc6-965e-cba83590229e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:32.808895 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc743b-b2a9-4cc6-965e-cba83590229e None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:32.809167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc743b-b2a9-4cc6-965e-cba83590229e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:32.809429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc743b-b2a9-4cc6-965e-cba83590229e None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:32.810132 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc743b-b2a9-4cc6-965e-cba83590229e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:32.814532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc743b-b2a9-4cc6-965e-cba83590229e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:32.816128 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc743b-b2a9-4cc6-965e-cba83590229e None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:32.837528 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 189/390] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:44:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:32.859624 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc743b-b2a9-4cc6-965e-cba83590229e None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:32.990368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3000be-9c45-41d6-82b7-5232274e4425 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:33.041883 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de43a6-ba4c-40df-a8be-77c54479f1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:33.042675 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de43a6-ba4c-40df-a8be-77c54479f1e7 None admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:33.042963 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de43a6-ba4c-40df-a8be-77c54479f1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:33.043245 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de43a6-ba4c-40df-a8be-77c54479f1e7 None admin] PATH_INFO: `/v3/projects/8d08421140694791b18a512ea16981a6/groups/b850e638b5bc42f7a7568111b339d0e1/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:33.045502 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33de43a6-ba4c-40df-a8be-77c54479f1e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d08421140694791b18a512ea16981a6, group_id=b850e638b5bc42f7a7568111b339d0e1, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:33.046977 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33de43a6-ba4c-40df-a8be-77c54479f1e7 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:33.050772 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:33.056191 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc743b-b2a9-4cc6-965e-cba83590229e None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:33.090751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99207bc3-44a1-48fc-8461-64ef0b0e49fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:33.091420 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99207bc3-44a1-48fc-8461-64ef0b0e49fa None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:33.091737 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99207bc3-44a1-48fc-8461-64ef0b0e49fa None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:33.091958 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99207bc3-44a1-48fc-8461-64ef0b0e49fa None admin] PATH_INFO: `/v3/groups/b850e638b5bc42f7a7568111b339d0e1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:33.095664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99207bc3-44a1-48fc-8461-64ef0b0e49fa None admin] RBAC: Authorizing `identity:get_group(group_id=b850e638b5bc42f7a7568111b339d0e1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:33.097336 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99207bc3-44a1-48fc-8461-64ef0b0e49fa None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:33.099469 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 190/391] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 31 10:44:32 2022] GET /identity/v3/groups/b850e638b5bc42f7a7568111b339d0e1 => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:44:33.101052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58444719-d389-4cb7-bb2c-85920928fcd6 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:33.101558 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58444719-d389-4cb7-bb2c-85920928fcd6 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:33.102046 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58444719-d389-4cb7-bb2c-85920928fcd6 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:33.108299 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 191/392] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:33.109481 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58444719-d389-4cb7-bb2c-85920928fcd6 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:33.119119 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 202/393] 10.222.0.16 () {58 vars in 1442 bytes} [Wed Aug 31 10:44:32 2022] PUT /identity/v3/projects/8d08421140694791b18a512ea16981a6/groups/b850e638b5bc42f7a7568111b339d0e1/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:44:33.124091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2e071-9388-4556-828c-b8ad1ff66070 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:33.124685 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2e071-9388-4556-828c-b8ad1ff66070 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:33.125114 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2e071-9388-4556-828c-b8ad1ff66070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:33.168595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-0ed2e071-9388-4556-828c-b8ad1ff66070 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:33.172756 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-0ed2e071-9388-4556-828c-b8ad1ff66070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:33.207313 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:33.316472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 203/394] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:33.337689 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58444719-d389-4cb7-bb2c-85920928fcd6 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:33.379073 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd1c996-feb8-4d90-ba3b-4d7b49c96047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:33.379967 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1c996-feb8-4d90-ba3b-4d7b49c96047 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:33.380467 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1c996-feb8-4d90-ba3b-4d7b49c96047 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:33.380881 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1c996-feb8-4d90-ba3b-4d7b49c96047 None admin] PATH_INFO: `/v3/projects/8d08421140694791b18a512ea16981a6` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:33.382242 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd1c996-feb8-4d90-ba3b-4d7b49c96047 None admin] RBAC: Authorizing `identity:get_project(project_id=8d08421140694791b18a512ea16981a6)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:33.383904 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd1c996-feb8-4d90-ba3b-4d7b49c96047 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:33.386154 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 192/395] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:33 2022] GET /identity/v3/projects/8d08421140694791b18a512ea16981a6 => generated 298 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:33.393670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2e071-9388-4556-828c-b8ad1ff66070 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:33.522867 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2e071-9388-4556-828c-b8ad1ff66070 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:33.562052 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7c52b-6d34-4e52-9fbd-86b5abd63709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:33.562805 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7c52b-6d34-4e52-9fbd-86b5abd63709 None admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:33.563084 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7c52b-6d34-4e52-9fbd-86b5abd63709 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:33.563361 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7c52b-6d34-4e52-9fbd-86b5abd63709 None admin] PATH_INFO: `/v3/projects/8d08421140694791b18a512ea16981a6/groups/b850e638b5bc42f7a7568111b339d0e1/roles/31d4b1dc082c41f7b6f18c338e7b51df` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:33.565521 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7c52b-6d34-4e52-9fbd-86b5abd63709 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d08421140694791b18a512ea16981a6, group_id=b850e638b5bc42f7a7568111b339d0e1, role_id=31d4b1dc082c41f7b6f18c338e7b51df)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:33.567112 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7c52b-6d34-4e52-9fbd-86b5abd63709 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:33.644166 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 204/396] 10.222.0.16 () {58 vars in 1442 bytes} [Wed Aug 31 10:44:33 2022] PUT /identity/v3/projects/8d08421140694791b18a512ea16981a6/groups/b850e638b5bc42f7a7568111b339d0e1/roles/31d4b1dc082c41f7b6f18c338e7b51df => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:44:33.650431 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd1c996-feb8-4d90-ba3b-4d7b49c96047 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:33.688248 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:33.788673 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eae62c-c5ff-486c-96f0-9533bc5c479a None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:33.789315 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eae62c-c5ff-486c-96f0-9533bc5c479a None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:33.789790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eae62c-c5ff-486c-96f0-9533bc5c479a None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:33.793778 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd1c996-feb8-4d90-ba3b-4d7b49c96047 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:33.795831 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 205/397] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:33.800952 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c271b88-8dd1-4904-a462-77a88eaf8884 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:33.801518 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c271b88-8dd1-4904-a462-77a88eaf8884 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:33.801994 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c271b88-8dd1-4904-a462-77a88eaf8884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:33.833035 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f12d06-69ab-43de-9b36-7fde6c1c9359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:33.833610 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f12d06-69ab-43de-9b36-7fde6c1c9359 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:33.833927 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f12d06-69ab-43de-9b36-7fde6c1c9359 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:33.834222 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f12d06-69ab-43de-9b36-7fde6c1c9359 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:33.835959 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-1c271b88-8dd1-4904-a462-77a88eaf8884 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:33.837402 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-1c271b88-8dd1-4904-a462-77a88eaf8884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:33.845667 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:33.846712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f12d06-69ab-43de-9b36-7fde6c1c9359 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:33.851655 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-e3f12d06-69ab-43de-9b36-7fde6c1c9359 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:44:33.851655 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:33.851655 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:33.851655 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:33.851655 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:33.851655 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:33.851655 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:33.851655 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:33.851655 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:33.851655 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:33.851655 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:33.851655 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:33.851655 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:33.851655 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:33.851655 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:33.851655 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:33.852586 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:33.852586 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:33.852586 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:33.852586 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:33.852586 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:33.852586 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:33.852586 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:33.852586 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:33.852586 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:33.852586 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:33.852586 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:33.852586 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:33.852586 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:33.852586 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:33.852586 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:33.852586 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:33.853413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:33.853413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:33.853413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:33.853413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:33.853413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:33.853413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:33.853413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:33.853413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:33.853413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:33.853413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:33.853413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:44:33.853413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:33.857384 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 193/398] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 31 10:44:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:33.859138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4406f6-c14f-4f48-b7ee-8525778014d3 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:33.860876 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4406f6-c14f-4f48-b7ee-8525778014d3 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:33.860876 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4406f6-c14f-4f48-b7ee-8525778014d3 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:33.861842 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 194/399] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:33.863585 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4406f6-c14f-4f48-b7ee-8525778014d3 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:33.938720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 206/400] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:33.947519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728ec91-ec6d-472b-a84d-3869ac3ffd61 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:33.947804 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728ec91-ec6d-472b-a84d-3869ac3ffd61 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:33.948073 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728ec91-ec6d-472b-a84d-3869ac3ffd61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:33.978746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:33.980695 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-4728ec91-ec6d-472b-a84d-3869ac3ffd61 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:33.982476 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-4728ec91-ec6d-472b-a84d-3869ac3ffd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:33.996204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4406f6-c14f-4f48-b7ee-8525778014d3 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:34.036626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b0612-b62d-411f-9495-511b5d7905ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:34.037859 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b0612-b62d-411f-9495-511b5d7905ee None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:34.038607 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b0612-b62d-411f-9495-511b5d7905ee None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:34.039119 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b0612-b62d-411f-9495-511b5d7905ee None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:34.039972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b0612-b62d-411f-9495-511b5d7905ee None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:34.040478 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b0612-b62d-411f-9495-511b5d7905ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:34.042075 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b0612-b62d-411f-9495-511b5d7905ee None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:34.052577 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 195/401] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:44:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:34.067530 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b0612-b62d-411f-9495-511b5d7905ee None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:34.137426 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 207/402] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:34.236683 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b0612-b62d-411f-9495-511b5d7905ee None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:34.271074 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4bf844-e83b-4ad3-ab6d-8362588894e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:34.271707 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bf844-e83b-4ad3-ab6d-8362588894e7 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:34.271980 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bf844-e83b-4ad3-ab6d-8362588894e7 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:34.272249 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bf844-e83b-4ad3-ab6d-8362588894e7 None admin] PATH_INFO: `/v3/groups/b850e638b5bc42f7a7568111b339d0e1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:34.273404 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4bf844-e83b-4ad3-ab6d-8362588894e7 None admin] RBAC: Authorizing `identity:get_group(group_id=b850e638b5bc42f7a7568111b339d0e1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:34.274881 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4bf844-e83b-4ad3-ab6d-8362588894e7 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:34.276853 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 196/403] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 31 10:44:34 2022] GET /identity/v3/groups/b850e638b5bc42f7a7568111b339d0e1 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:34.282971 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4728ec91-ec6d-472b-a84d-3869ac3ffd61 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:34.334844 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4bf844-e83b-4ad3-ab6d-8362588894e7 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:34.362763 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:34.431684 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:34.462886 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4bf844-e83b-4ad3-ab6d-8362588894e7 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:34.501305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924ad6b-caf5-403f-935e-1cb5ac3cf78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:34.502447 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924ad6b-caf5-403f-935e-1cb5ac3cf78f None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:34.502947 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924ad6b-caf5-403f-935e-1cb5ac3cf78f None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:34.503405 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924ad6b-caf5-403f-935e-1cb5ac3cf78f None admin] PATH_INFO: `/v3/roles/member` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:34.516547 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9924ad6b-caf5-403f-935e-1cb5ac3cf78f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:34.521669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-9924ad6b-caf5-403f-935e-1cb5ac3cf78f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:44:34.521669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:34.521669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:34.521669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:34.521669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:34.521669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:34.521669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:34.521669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:34.521669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:34.521669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:34.521669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:34.521669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:34.521669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:34.521669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:34.521669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:34.521669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:34.522599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:34.522599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:34.522599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:34.522599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:34.522599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:34.522599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:34.522599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:34.522599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:34.522599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:34.522599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:34.522599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:34.522599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:34.522599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:34.522599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:34.522599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:34.522599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:34.523707 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:34.523707 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:34.523707 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:34.523707 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:34.523707 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:34.523707 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:34.523707 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:34.523707 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:34.523707 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:34.523707 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:34.523707 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:44:34.523707 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:34.523707 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 197/404] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:44:34.524352 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924ad6b-caf5-403f-935e-1cb5ac3cf78f None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:34.526271 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4728ec91-ec6d-472b-a84d-3869ac3ffd61 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:34.570871 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f467e-9952-4b06-871e-d9b270552a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:34.571832 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f467e-9952-4b06-871e-d9b270552a82 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:34.572393 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f467e-9952-4b06-871e-d9b270552a82 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:34.572853 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f467e-9952-4b06-871e-d9b270552a82 None admin] PATH_INFO: `/v3/projects/08b4540d439e44adb9398aec4130989a` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:34.574246 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f467e-9952-4b06-871e-d9b270552a82 None admin] RBAC: Authorizing `identity:get_project(project_id=08b4540d439e44adb9398aec4130989a)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:34.576229 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f467e-9952-4b06-871e-d9b270552a82 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:34.578613 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 208/405] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:34 2022] GET /identity/v3/projects/08b4540d439e44adb9398aec4130989a => generated 302 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:34.579664 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f467e-9952-4b06-871e-d9b270552a82 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:34.619075 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:34.645353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924ad6b-caf5-403f-935e-1cb5ac3cf78f None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:34.685932 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97745f-48eb-4359-ad2a-e8606d86c20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:34.686939 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97745f-48eb-4359-ad2a-e8606d86c20b None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:34.686939 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97745f-48eb-4359-ad2a-e8606d86c20b None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:34.687113 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97745f-48eb-4359-ad2a-e8606d86c20b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:34.701879 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97745f-48eb-4359-ad2a-e8606d86c20b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:34.708882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-5b97745f-48eb-4359-ad2a-e8606d86c20b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:44:34.708882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:34.708882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:34.708882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:34.708882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:34.708882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:34.708882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:34.708882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:34.708882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:34.708882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:34.708882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:34.708882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:34.708882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:34.708882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:34.708882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:34.708882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:34.709816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:34.709816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:34.709816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:34.709816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:34.709816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:34.709816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:34.709816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:34.709816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:34.709816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:34.709816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:34.709816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:34.709816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:34.709816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:34.709816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:34.709816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:34.709816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:34.710865 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:34.710865 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:34.710865 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:34.710865 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:34.710865 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:34.710865 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:34.710865 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:34.710865 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:34.710865 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:34.710865 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:34.710865 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:44:34.710865 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:34.710865 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 198/406] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:44:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:34.711899 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97745f-48eb-4359-ad2a-e8606d86c20b None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:34.712276 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f467e-9952-4b06-871e-d9b270552a82 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:34.764169 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c81218-a7f5-408e-8330-cef43ca49602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:34.764809 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c81218-a7f5-408e-8330-cef43ca49602 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:34.765082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c81218-a7f5-408e-8330-cef43ca49602 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:34.765351 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c81218-a7f5-408e-8330-cef43ca49602 None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:34.766007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c81218-a7f5-408e-8330-cef43ca49602 None admin] RBAC: Adding query filter params (name=member) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:34.766306 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c81218-a7f5-408e-8330-cef43ca49602 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:34.767768 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c81218-a7f5-408e-8330-cef43ca49602 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:34.770674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:34.781332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 209/407] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 31 10:44:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:34.782125 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c81218-a7f5-408e-8330-cef43ca49602 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:34.862021 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97745f-48eb-4359-ad2a-e8606d86c20b None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:34.909781 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb2ad0-f8ab-4cc7-a259-e53f0902da91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:34.910834 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb2ad0-f8ab-4cc7-a259-e53f0902da91 None admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:34.911016 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c81218-a7f5-408e-8330-cef43ca49602 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:34.911763 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb2ad0-f8ab-4cc7-a259-e53f0902da91 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:34.912454 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb2ad0-f8ab-4cc7-a259-e53f0902da91 None admin] PATH_INFO: `/v3/projects/08b4540d439e44adb9398aec4130989a/groups/b850e638b5bc42f7a7568111b339d0e1/roles/31d4b1dc082c41f7b6f18c338e7b51df` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:34.916426 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb2ad0-f8ab-4cc7-a259-e53f0902da91 None admin] RBAC: Authorizing `identity:create_grant(project_id=08b4540d439e44adb9398aec4130989a, group_id=b850e638b5bc42f7a7568111b339d0e1, role_id=31d4b1dc082c41f7b6f18c338e7b51df)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:34.919210 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb2ad0-f8ab-4cc7-a259-e53f0902da91 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:34.950054 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a185a4d-73c3-4601-aa60-6e850169d312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:34.951473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a185a4d-73c3-4601-aa60-6e850169d312 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:34.951753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a185a4d-73c3-4601-aa60-6e850169d312 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:34.952017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a185a4d-73c3-4601-aa60-6e850169d312 None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:34.952683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a185a4d-73c3-4601-aa60-6e850169d312 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:34.953038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a185a4d-73c3-4601-aa60-6e850169d312 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:34.954498 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a185a4d-73c3-4601-aa60-6e850169d312 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:34.965193 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 210/408] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:34.965774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a185a4d-73c3-4601-aa60-6e850169d312 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:35.051049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:35.062566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 199/409] 10.222.0.16 () {58 vars in 1442 bytes} [Wed Aug 31 10:44:34 2022] PUT /identity/v3/projects/08b4540d439e44adb9398aec4130989a/groups/b850e638b5bc42f7a7568111b339d0e1/roles/31d4b1dc082c41f7b6f18c338e7b51df => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:44:35.064495 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb2ad0-f8ab-4cc7-a259-e53f0902da91 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:35.082524 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:35.162106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a185a4d-73c3-4601-aa60-6e850169d312 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:35.212688 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ce6e4-6f70-4930-9b0d-0c02c94e9564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:35.213344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ce6e4-6f70-4930-9b0d-0c02c94e9564 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:35.213622 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ce6e4-6f70-4930-9b0d-0c02c94e9564 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:35.213998 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ce6e4-6f70-4930-9b0d-0c02c94e9564 None admin] PATH_INFO: `/v3/groups/b850e638b5bc42f7a7568111b339d0e1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:35.215696 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ce6e4-6f70-4930-9b0d-0c02c94e9564 None admin] RBAC: Authorizing `identity:get_group(group_id=b850e638b5bc42f7a7568111b339d0e1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:35.217752 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ce6e4-6f70-4930-9b0d-0c02c94e9564 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:35.220367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 211/410] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 31 10:44:34 2022] GET /identity/v3/groups/b850e638b5bc42f7a7568111b339d0e1 => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:35.228234 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ce6e4-6f70-4930-9b0d-0c02c94e9564 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:35.363998 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb2ad0-f8ab-4cc7-a259-e53f0902da91 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:35.395395 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ce6e4-6f70-4930-9b0d-0c02c94e9564 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:35.416239 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd71962-c952-456a-a66a-f96549257625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:35.417787 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd71962-c952-456a-a66a-f96549257625 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:35.418558 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd71962-c952-456a-a66a-f96549257625 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:35.420022 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd71962-c952-456a-a66a-f96549257625 None admin] PATH_INFO: `/v3/groups/13cb05d6493d46898a644db37bd7a699` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:35.422624 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd71962-c952-456a-a66a-f96549257625 None admin] RBAC: Authorizing `identity:get_group(group_id=13cb05d6493d46898a644db37bd7a699)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:35.423568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:35.426882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd71962-c952-456a-a66a-f96549257625 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:35.430758 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 200/411] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 31 10:44:35 2022] GET /identity/v3/groups/13cb05d6493d46898a644db37bd7a699 => generated 229 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:35.434499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59936c95-a91b-470e-bf9f-943de273ea68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:35.435165 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59936c95-a91b-470e-bf9f-943de273ea68 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:35.435437 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59936c95-a91b-470e-bf9f-943de273ea68 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:35.435984 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59936c95-a91b-470e-bf9f-943de273ea68 None admin] PATH_INFO: `/v3/projects/08b4540d439e44adb9398aec4130989a` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:35.437074 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59936c95-a91b-470e-bf9f-943de273ea68 None admin] RBAC: Authorizing `identity:get_project(project_id=08b4540d439e44adb9398aec4130989a)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:35.437576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd71962-c952-456a-a66a-f96549257625 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:35.438618 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59936c95-a91b-470e-bf9f-943de273ea68 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:35.440670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 212/412] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:35 2022] GET /identity/v3/projects/08b4540d439e44adb9398aec4130989a => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:44:35.446173 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59936c95-a91b-470e-bf9f-943de273ea68 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:35.552476 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:35.589577 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd71962-c952-456a-a66a-f96549257625 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:35.589797 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59936c95-a91b-470e-bf9f-943de273ea68 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:35.633015 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe2caf-d993-4196-bd03-dfbd3b9829f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:35.633791 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe2caf-d993-4196-bd03-dfbd3b9829f7 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:35.634064 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe2caf-d993-4196-bd03-dfbd3b9829f7 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:35.634333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe2caf-d993-4196-bd03-dfbd3b9829f7 None admin] PATH_INFO: `/v3/projects/776925ef060044f49d8dab7d8928f73c` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:35.635568 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fe2caf-d993-4196-bd03-dfbd3b9829f7 None admin] RBAC: Authorizing `identity:get_project(project_id=776925ef060044f49d8dab7d8928f73c)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:35.637773 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fe2caf-d993-4196-bd03-dfbd3b9829f7 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:35.640380 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 201/413] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:35 2022] GET /identity/v3/projects/776925ef060044f49d8dab7d8928f73c => generated 344 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:35.641817 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273f5c7-9227-4d51-878e-ea968ed1afda None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:35.642093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273f5c7-9227-4d51-878e-ea968ed1afda None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:35.642361 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273f5c7-9227-4d51-878e-ea968ed1afda None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:35.643645 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 202/414] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:35.647022 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273f5c7-9227-4d51-878e-ea968ed1afda None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:35.660877 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40cb7a9-21e7-4415-a64c-93b850815774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:35.661987 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40cb7a9-21e7-4415-a64c-93b850815774 None admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:35.662273 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40cb7a9-21e7-4415-a64c-93b850815774 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:35.662578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40cb7a9-21e7-4415-a64c-93b850815774 None admin] PATH_INFO: `/v3/projects/08b4540d439e44adb9398aec4130989a/groups/b850e638b5bc42f7a7568111b339d0e1/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:35.665266 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40cb7a9-21e7-4415-a64c-93b850815774 None admin] RBAC: Authorizing `identity:create_grant(project_id=08b4540d439e44adb9398aec4130989a, group_id=b850e638b5bc42f7a7568111b339d0e1, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:35.667645 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40cb7a9-21e7-4415-a64c-93b850815774 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:35.746506 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 213/415] 10.222.0.16 () {58 vars in 1442 bytes} [Wed Aug 31 10:44:35 2022] PUT /identity/v3/projects/08b4540d439e44adb9398aec4130989a/groups/b850e638b5bc42f7a7568111b339d0e1/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 10:44:35.748112 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4c66d-f0a2-4577-a530-baff917b7675 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:35.748393 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4c66d-f0a2-4577-a530-baff917b7675 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:35.748661 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4c66d-f0a2-4577-a530-baff917b7675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:35.778181 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273f5c7-9227-4d51-878e-ea968ed1afda None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:35.790564 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-02c4c66d-f0a2-4577-a530-baff917b7675 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:35.791785 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-02c4c66d-f0a2-4577-a530-baff917b7675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:35.848497 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8afcbd-30eb-4eac-9287-f7cf5c19e738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:35.849178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8afcbd-30eb-4eac-9287-f7cf5c19e738 None admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:35.849450 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8afcbd-30eb-4eac-9287-f7cf5c19e738 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:35.849717 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8afcbd-30eb-4eac-9287-f7cf5c19e738 None admin] PATH_INFO: `/v3/projects/776925ef060044f49d8dab7d8928f73c/groups/13cb05d6493d46898a644db37bd7a699/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:35.853839 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8afcbd-30eb-4eac-9287-f7cf5c19e738 None admin] RBAC: Authorizing `identity:create_grant(project_id=776925ef060044f49d8dab7d8928f73c, group_id=13cb05d6493d46898a644db37bd7a699, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:35.855338 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8afcbd-30eb-4eac-9287-f7cf5c19e738 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:35.858705 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:35.895026 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:35.914254 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 214/416] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:35.971489 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 203/417] 10.222.0.16 () {58 vars in 1442 bytes} [Wed Aug 31 10:44:35 2022] PUT /identity/v3/projects/776925ef060044f49d8dab7d8928f73c/groups/13cb05d6493d46898a644db37bd7a699/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:44:36.175283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876add8-22aa-4a9b-87f6-d9f705b66f0f None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:36.175643 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876add8-22aa-4a9b-87f6-d9f705b66f0f None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:36.175833 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876add8-22aa-4a9b-87f6-d9f705b66f0f None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:36.177463 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 215/418] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:36.182954 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54155efa-ed9b-4648-a3a9-01412c1e6854 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:36.183232 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54155efa-ed9b-4648-a3a9-01412c1e6854 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:36.183498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54155efa-ed9b-4648-a3a9-01412c1e6854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:36.201860 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876add8-22aa-4a9b-87f6-d9f705b66f0f None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:36.215575 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-54155efa-ed9b-4648-a3a9-01412c1e6854 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:36.217280 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-54155efa-ed9b-4648-a3a9-01412c1e6854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:36.314379 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 204/419] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:44:36.379071 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876add8-22aa-4a9b-87f6-d9f705b66f0f None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:36.423493 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671634f-8a07-4caf-99bf-c4a03036442d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:36.424905 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671634f-8a07-4caf-99bf-c4a03036442d None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:36.425535 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671634f-8a07-4caf-99bf-c4a03036442d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:36.426027 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671634f-8a07-4caf-99bf-c4a03036442d None admin] PATH_INFO: `/v3/roles/member` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:36.438866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5671634f-8a07-4caf-99bf-c4a03036442d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:36.444250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-5671634f-8a07-4caf-99bf-c4a03036442d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:44:36.444250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:36.444250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:36.444250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:36.444250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:36.444250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:36.444250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:36.444250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:36.444250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:36.444250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:36.444250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:36.444250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:36.444250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:36.444250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:36.444250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:36.444250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:36.445098 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:36.445098 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:36.445098 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:36.445098 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:36.445098 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:36.445098 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:36.445098 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:36.445098 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:36.445098 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:36.445098 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:36.445098 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:36.445098 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:36.445098 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:36.445098 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:36.445098 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:36.445098 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:36.445879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:36.445879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:36.445879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:36.445879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:36.445879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:36.445879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:36.445879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:36.445879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:36.445879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:36.445879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:36.445879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:44:36.445879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:36.448714 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 216/420] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:36.459526 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54155efa-ed9b-4648-a3a9-01412c1e6854 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:36.599838 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54155efa-ed9b-4648-a3a9-01412c1e6854 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:36.631235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:36.650193 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe97b2-121e-4be1-82b9-9bd4df479746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:36.650835 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe97b2-121e-4be1-82b9-9bd4df479746 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:36.651112 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe97b2-121e-4be1-82b9-9bd4df479746 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:36.651384 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe97b2-121e-4be1-82b9-9bd4df479746 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:36.652054 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfe97b2-121e-4be1-82b9-9bd4df479746 None admin] RBAC: Adding query filter params (name=member) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:36.652354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfe97b2-121e-4be1-82b9-9bd4df479746 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:36.654246 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfe97b2-121e-4be1-82b9-9bd4df479746 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:36.662926 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 205/421] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 31 10:44:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:36.668446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671634f-8a07-4caf-99bf-c4a03036442d None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:36.678523 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe97b2-121e-4be1-82b9-9bd4df479746 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:36.733198 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:36.838274 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671634f-8a07-4caf-99bf-c4a03036442d None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:36.875790 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe97b2-121e-4be1-82b9-9bd4df479746 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:36.918142 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509feeb-72da-450d-a306-29e92ad6e19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:36.919159 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509feeb-72da-450d-a306-29e92ad6e19b None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:36.919686 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509feeb-72da-450d-a306-29e92ad6e19b None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:36.920154 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509feeb-72da-450d-a306-29e92ad6e19b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:36.951273 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae4da0-f905-4a52-b124-26daaf43fc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:36.953602 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae4da0-f905-4a52-b124-26daaf43fc5e None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:36.955459 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae4da0-f905-4a52-b124-26daaf43fc5e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:36.955962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae4da0-f905-4a52-b124-26daaf43fc5e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:36.978808 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dae4da0-f905-4a52-b124-26daaf43fc5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:36.988053 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-3dae4da0-f905-4a52-b124-26daaf43fc5e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:44:36.988053 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:36.988053 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:36.988053 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:36.988053 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:36.988053 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:36.988053 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:36.988053 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:36.988053 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:36.988053 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:36.988053 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:36.988053 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:36.988053 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:36.988053 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:36.988053 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:36.988053 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:36.989042 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:36.989042 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:36.989042 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:36.989042 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:36.989042 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:36.989042 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:36.989042 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:36.989042 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:36.989042 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:36.989042 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:36.989042 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:36.989042 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:36.989042 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:36.989042 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:36.989042 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:36.989042 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:36.989824 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:36.989824 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:36.989824 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:36.989824 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:36.989824 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:36.989824 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:36.989824 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:36.989824 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:36.989824 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:36.989824 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:36.989824 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:44:36.989824 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:36.993448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 206/422] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 31 10:44:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:37.000088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae4da0-f905-4a52-b124-26daaf43fc5e None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:37.031252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d509feeb-72da-450d-a306-29e92ad6e19b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:37.032938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d509feeb-72da-450d-a306-29e92ad6e19b None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:37.098335 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:37.110169 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:37.203539 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 217/423] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:44:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:44:37.208899 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509feeb-72da-450d-a306-29e92ad6e19b None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:37.248363 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae4da0-f905-4a52-b124-26daaf43fc5e None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:37.321240 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509feeb-72da-450d-a306-29e92ad6e19b None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:37.359298 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268414d-811f-4c84-a6bd-2cfe5fb5dc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:37.360262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268414d-811f-4c84-a6bd-2cfe5fb5dc6f None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:37.360731 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268414d-811f-4c84-a6bd-2cfe5fb5dc6f None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:37.361190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268414d-811f-4c84-a6bd-2cfe5fb5dc6f None admin] PATH_INFO: `/v3/projects/8d08421140694791b18a512ea16981a6` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:37.362573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4268414d-811f-4c84-a6bd-2cfe5fb5dc6f None admin] RBAC: Authorizing `identity:get_project(project_id=8d08421140694791b18a512ea16981a6)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:37.364382 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4268414d-811f-4c84-a6bd-2cfe5fb5dc6f None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:37.366542 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 218/424] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:37 2022] GET /identity/v3/projects/8d08421140694791b18a512ea16981a6 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:37.371763 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268414d-811f-4c84-a6bd-2cfe5fb5dc6f None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:37.376891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60f9a9-bc3b-465a-8f1e-072dc1bd548c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:37.377767 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60f9a9-bc3b-465a-8f1e-072dc1bd548c None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:37.379998 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60f9a9-bc3b-465a-8f1e-072dc1bd548c None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:37.380487 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60f9a9-bc3b-465a-8f1e-072dc1bd548c None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:37.384711 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f60f9a9-bc3b-465a-8f1e-072dc1bd548c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:37.385222 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f60f9a9-bc3b-465a-8f1e-072dc1bd548c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:37.386840 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f60f9a9-bc3b-465a-8f1e-072dc1bd548c None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:37.402754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:37.407341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 207/425] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:44:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:37.412873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60f9a9-bc3b-465a-8f1e-072dc1bd548c None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:37.527635 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268414d-811f-4c84-a6bd-2cfe5fb5dc6f None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:37.545125 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60f9a9-bc3b-465a-8f1e-072dc1bd548c None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:37.554873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:37.571866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ed900-5e13-429c-8faa-a16e2538e7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:37.573504 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ed900-5e13-429c-8faa-a16e2538e7ba None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:37.574272 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ed900-5e13-429c-8faa-a16e2538e7ba None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:37.574797 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ed900-5e13-429c-8faa-a16e2538e7ba None admin] PATH_INFO: `/v3/groups/b850e638b5bc42f7a7568111b339d0e1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:37.577933 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ed900-5e13-429c-8faa-a16e2538e7ba None admin] RBAC: Authorizing `identity:get_group(group_id=b850e638b5bc42f7a7568111b339d0e1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:37.580367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ed900-5e13-429c-8faa-a16e2538e7ba None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:37.581792 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f19d5-e7f8-4924-ac9d-59eeae397d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:37.582532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f19d5-e7f8-4924-ac9d-59eeae397d23 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:37.583007 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f19d5-e7f8-4924-ac9d-59eeae397d23 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:37.583461 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 219/426] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 31 10:44:37 2022] GET /identity/v3/groups/b850e638b5bc42f7a7568111b339d0e1 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:37.584963 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3ba06-89f3-4f75-bff5-327b25daa858 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:37.585417 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3ba06-89f3-4f75-bff5-327b25daa858 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:37.585843 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3ba06-89f3-4f75-bff5-327b25daa858 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:37.586787 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f19d5-e7f8-4924-ac9d-59eeae397d23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:37.588100 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 220/427] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:37.589867 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e3ba06-89f3-4f75-bff5-327b25daa858 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:37.682942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f19d5-e7f8-4924-ac9d-59eeae397d23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:37.686144 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f19d5-e7f8-4924-ac9d-59eeae397d23 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:37.693069 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:37.722270 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e3ba06-89f3-4f75-bff5-327b25daa858 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:37.770895 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf68451-545e-4649-91a8-e2c437b175b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:37.771891 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf68451-545e-4649-91a8-e2c437b175b9 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:37.772823 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf68451-545e-4649-91a8-e2c437b175b9 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:37.773280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf68451-545e-4649-91a8-e2c437b175b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:37.774632 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf68451-545e-4649-91a8-e2c437b175b9 None admin] RBAC: Adding query filter params (group.id=b850e638b5bc42f7a7568111b339d0e1, role.id=2278da3c52644a819d210b5f9c20b6c1, scope.project.id=8d08421140694791b18a512ea16981a6) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:37.775406 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf68451-545e-4649-91a8-e2c437b175b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:37.777336 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf68451-545e-4649-91a8-e2c437b175b9 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:37.786248 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 221/428] 10.222.0.16 () {56 vars in 1376 bytes} [Wed Aug 31 10:44:37 2022] GET /identity/v3/role_assignments?group.id=b850e638b5bc42f7a7568111b339d0e1&scope.project.id=8d08421140694791b18a512ea16981a6&role.id=2278da3c52644a819d210b5f9c20b6c1 => generated 611 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:44:37.787667 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e56802-7bca-425b-b0c2-d41ffee85c0d None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:37.788121 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e56802-7bca-425b-b0c2-d41ffee85c0d None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:37.788585 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e56802-7bca-425b-b0c2-d41ffee85c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:37.812268 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-a5e56802-7bca-425b-b0c2-d41ffee85c0d None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:37.814066 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a5e56802-7bca-425b-b0c2-d41ffee85c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:37.816156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 208/429] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:44:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:44:37.822214 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f19d5-e7f8-4924-ac9d-59eeae397d23 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:37.839831 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:37.968751 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 222/430] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:38.002950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f19d5-e7f8-4924-ac9d-59eeae397d23 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:38.039931 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085f236-972f-4429-acb6-72e6f756ca7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:38.040933 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085f236-972f-4429-acb6-72e6f756ca7e None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:38.041426 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085f236-972f-4429-acb6-72e6f756ca7e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:38.041902 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085f236-972f-4429-acb6-72e6f756ca7e None admin] PATH_INFO: `/v3/projects/8d08421140694791b18a512ea16981a6` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:38.043288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b085f236-972f-4429-acb6-72e6f756ca7e None admin] RBAC: Authorizing `identity:get_project(project_id=8d08421140694791b18a512ea16981a6)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:38.045088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b085f236-972f-4429-acb6-72e6f756ca7e None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:38.047270 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 209/431] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:37 2022] GET /identity/v3/projects/8d08421140694791b18a512ea16981a6 => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:44:38.052836 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e56802-7bca-425b-b0c2-d41ffee85c0d None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:38.089708 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4b5e8-5d17-4780-b4f9-83cdd0cf0012 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:38.090343 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4b5e8-5d17-4780-b4f9-83cdd0cf0012 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:38.091120 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4b5e8-5d17-4780-b4f9-83cdd0cf0012 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:38.093022 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 210/432] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:44:38.098515 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7f33f-ad14-418a-98f7-cf43554d4fd6 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:38.099077 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7f33f-ad14-418a-98f7-cf43554d4fd6 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:38.099546 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7f33f-ad14-418a-98f7-cf43554d4fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:38.119587 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-18b7f33f-ad14-418a-98f7-cf43554d4fd6 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:38.121262 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-18b7f33f-ad14-418a-98f7-cf43554d4fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:38.149564 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:38.177876 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e56802-7bca-425b-b0c2-d41ffee85c0d None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:38.201500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 211/433] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:44:38.221569 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14648870-49f4-4e1d-bf8b-c1b2531c7848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:38.222537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14648870-49f4-4e1d-bf8b-c1b2531c7848 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:38.223021 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14648870-49f4-4e1d-bf8b-c1b2531c7848 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:38.223467 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14648870-49f4-4e1d-bf8b-c1b2531c7848 None admin] PATH_INFO: `/v3/groups/b850e638b5bc42f7a7568111b339d0e1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:38.225184 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14648870-49f4-4e1d-bf8b-c1b2531c7848 None admin] RBAC: Authorizing `identity:get_group(group_id=b850e638b5bc42f7a7568111b339d0e1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:38.226863 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14648870-49f4-4e1d-bf8b-c1b2531c7848 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:38.229030 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 223/434] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 31 10:44:38 2022] GET /identity/v3/groups/b850e638b5bc42f7a7568111b339d0e1 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:38.234765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7f33f-ad14-418a-98f7-cf43554d4fd6 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:38.246716 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:38.291465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14648870-49f4-4e1d-bf8b-c1b2531c7848 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:38.377463 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7f33f-ad14-418a-98f7-cf43554d4fd6 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:38.426902 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e8497e-b991-491d-bd70-1c69df035d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:38.427636 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8497e-b991-491d-bd70-1c69df035d1a None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:38.427911 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8497e-b991-491d-bd70-1c69df035d1a None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:38.428294 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8497e-b991-491d-bd70-1c69df035d1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:38.429025 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e8497e-b991-491d-bd70-1c69df035d1a None admin] RBAC: Adding query filter params (group.id=b850e638b5bc42f7a7568111b339d0e1, role.id=31d4b1dc082c41f7b6f18c338e7b51df, scope.project.id=8d08421140694791b18a512ea16981a6) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:38.429331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e8497e-b991-491d-bd70-1c69df035d1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:38.430908 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e8497e-b991-491d-bd70-1c69df035d1a None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:38.438381 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 212/435] 10.222.0.16 () {56 vars in 1376 bytes} [Wed Aug 31 10:44:38 2022] GET /identity/v3/role_assignments?group.id=b850e638b5bc42f7a7568111b339d0e1&scope.project.id=8d08421140694791b18a512ea16981a6&role.id=31d4b1dc082c41f7b6f18c338e7b51df => generated 611 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:38.438976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14648870-49f4-4e1d-bf8b-c1b2531c7848 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:38.486207 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78907cfb-190f-4e3b-beab-c9fd4bcd5783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:38.487185 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78907cfb-190f-4e3b-beab-c9fd4bcd5783 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:38.487673 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78907cfb-190f-4e3b-beab-c9fd4bcd5783 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:38.488135 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78907cfb-190f-4e3b-beab-c9fd4bcd5783 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:38.502601 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f5d92-d386-4c27-b9a4-e5531a59268b None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:38.502860 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f5d92-d386-4c27-b9a4-e5531a59268b None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:38.503107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f5d92-d386-4c27-b9a4-e5531a59268b None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:38.504516 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 213/436] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:44:38.506347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78907cfb-190f-4e3b-beab-c9fd4bcd5783 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:38.510441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fa47e-075e-475e-a619-71d0283c13f0 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:38.510734 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fa47e-075e-475e-a619-71d0283c13f0 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:38.511010 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fa47e-075e-475e-a619-71d0283c13f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:38.513217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-78907cfb-190f-4e3b-beab-c9fd4bcd5783 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:44:38.513217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:38.513217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:38.513217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:38.513217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:38.513217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:38.513217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:38.513217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:38.513217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:38.513217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:38.513217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:38.513217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:38.513217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:38.513217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:38.513217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:38.513217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:38.514264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:38.514264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:38.514264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:38.514264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:38.514264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:38.514264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:38.514264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:38.514264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:38.514264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:38.514264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:38.514264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:38.514264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:38.514264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:38.514264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:38.514264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:38.514264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:38.515105 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:38.515105 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:38.515105 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:38.515105 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:38.515105 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:38.515105 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:38.515105 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:38.515105 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:38.515105 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:38.515105 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:38.515105 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:44:38.515105 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:38.515105 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 224/437] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 31 10:44:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:44:38.520635 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78907cfb-190f-4e3b-beab-c9fd4bcd5783 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:38.534531 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:38.536990 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-c45fa47e-075e-475e-a619-71d0283c13f0 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:38.538243 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-c45fa47e-075e-475e-a619-71d0283c13f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:38.637878 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:38.643540 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 214/438] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:38.644417 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fa47e-075e-475e-a619-71d0283c13f0 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:38.670277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78907cfb-190f-4e3b-beab-c9fd4bcd5783 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:38.714206 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c6df7-b35b-47e3-adc3-b2ba75ebe4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:38.715040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c6df7-b35b-47e3-adc3-b2ba75ebe4de None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:38.715322 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c6df7-b35b-47e3-adc3-b2ba75ebe4de None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:38.715593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c6df7-b35b-47e3-adc3-b2ba75ebe4de None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:38.716584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c6df7-b35b-47e3-adc3-b2ba75ebe4de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:38.717089 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c6df7-b35b-47e3-adc3-b2ba75ebe4de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:38.718538 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c6df7-b35b-47e3-adc3-b2ba75ebe4de None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:38.727897 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 225/439] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:44:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:38.736718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c6df7-b35b-47e3-adc3-b2ba75ebe4de None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:38.780006 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fa47e-075e-475e-a619-71d0283c13f0 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:38.821455 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:38.827338 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d56b8-dabd-40e2-9a1a-4a01917eb3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:38.827987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d56b8-dabd-40e2-9a1a-4a01917eb3d6 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:38.828264 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d56b8-dabd-40e2-9a1a-4a01917eb3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:38.828533 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d56b8-dabd-40e2-9a1a-4a01917eb3d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:38.843508 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4d56b8-dabd-40e2-9a1a-4a01917eb3d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:38.849362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-9d4d56b8-dabd-40e2-9a1a-4a01917eb3d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:44:38.849362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:38.849362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:38.849362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:38.849362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:38.849362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:38.849362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:38.849362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:38.849362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:38.849362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:38.849362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:38.849362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:38.849362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:38.849362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:38.849362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:38.849362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:38.850301 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:38.850301 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:38.850301 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:38.850301 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:38.850301 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:38.850301 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:38.850301 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:38.850301 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:38.850301 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:38.850301 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:38.850301 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:38.850301 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:38.850301 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:38.850301 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:38.850301 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:38.850301 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:38.851780 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:38.851780 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:38.851780 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:38.851780 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:38.851780 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:38.851780 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:38.851780 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:38.851780 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:38.851780 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:38.851780 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:38.851780 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:44:38.851780 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:38.851780 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 215/440] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:38.859551 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d56b8-dabd-40e2-9a1a-4a01917eb3d6 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:38.865525 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c6df7-b35b-47e3-adc3-b2ba75ebe4de None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:38.906945 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9959d0-4c25-43b8-a555-4cad5c50b626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:38.907474 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9959d0-4c25-43b8-a555-4cad5c50b626 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:38.907915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9959d0-4c25-43b8-a555-4cad5c50b626 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:38.908387 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9959d0-4c25-43b8-a555-4cad5c50b626 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:38.937098 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:38.982594 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d56b8-dabd-40e2-9a1a-4a01917eb3d6 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:38.988518 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9959d0-4c25-43b8-a555-4cad5c50b626 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:38.989984 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9959d0-4c25-43b8-a555-4cad5c50b626 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:39.018362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7a0f0-797f-46a2-98a3-7dcffb73918b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:39.019002 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7a0f0-797f-46a2-98a3-7dcffb73918b None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:39.019274 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7a0f0-797f-46a2-98a3-7dcffb73918b None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:39.019540 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7a0f0-797f-46a2-98a3-7dcffb73918b None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:39.020205 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7a0f0-797f-46a2-98a3-7dcffb73918b None admin] RBAC: Adding query filter params (name=member) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:39.020503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7a0f0-797f-46a2-98a3-7dcffb73918b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:39.021892 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7a0f0-797f-46a2-98a3-7dcffb73918b None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:39.030452 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 216/441] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 31 10:44:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:39.030452 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7a0f0-797f-46a2-98a3-7dcffb73918b None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:39.063018 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:39.099357 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 226/442] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:44:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:44:39.100537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9959d0-4c25-43b8-a555-4cad5c50b626 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:39.139369 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7a0f0-797f-46a2-98a3-7dcffb73918b None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:39.162917 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:39.175798 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef2fed-04a1-4689-b8ba-2a3631c4e58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:39.176440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef2fed-04a1-4689-b8ba-2a3631c4e58b None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:39.176714 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef2fed-04a1-4689-b8ba-2a3631c4e58b None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:39.176984 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef2fed-04a1-4689-b8ba-2a3631c4e58b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:39.188074 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef2fed-04a1-4689-b8ba-2a3631c4e58b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:39.192803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-8cef2fed-04a1-4689-b8ba-2a3631c4e58b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:44:39.192803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:39.192803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:39.192803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:39.192803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:39.192803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:39.192803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:39.192803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:39.192803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:39.192803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:39.192803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:39.192803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:39.192803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:39.192803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:39.192803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:39.192803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:39.193765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:39.193765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:39.193765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:39.193765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:39.193765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:39.193765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:39.193765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:39.193765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:39.193765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:39.193765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:39.193765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:39.193765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:39.193765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:39.193765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:39.193765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:39.193765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:39.195627 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:39.195627 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:39.195627 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:39.195627 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:39.195627 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:39.195627 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:39.195627 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:39.195627 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:39.195627 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:39.195627 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:39.195627 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:44:39.195627 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:39.195627 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 217/443] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:44:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:39.195627 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef2fed-04a1-4689-b8ba-2a3631c4e58b None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:39.205738 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9959d0-4c25-43b8-a555-4cad5c50b626 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:39.239439 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd57325-4f07-4611-abc0-2b078c5c0994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:39.240044 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd57325-4f07-4611-abc0-2b078c5c0994 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:39.240320 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd57325-4f07-4611-abc0-2b078c5c0994 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:39.240591 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd57325-4f07-4611-abc0-2b078c5c0994 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:39.274679 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:39.297873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef2fed-04a1-4689-b8ba-2a3631c4e58b None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:39.311378 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd57325-4f07-4611-abc0-2b078c5c0994 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:39.312883 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd57325-4f07-4611-abc0-2b078c5c0994 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:39.332281 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bfd76-7e01-412c-bf75-306cb5f8a49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:39.333060 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bfd76-7e01-412c-bf75-306cb5f8a49e None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:39.333398 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bfd76-7e01-412c-bf75-306cb5f8a49e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:39.333736 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bfd76-7e01-412c-bf75-306cb5f8a49e None admin] PATH_INFO: `/v3/projects/08b4540d439e44adb9398aec4130989a` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:39.334906 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110bfd76-7e01-412c-bf75-306cb5f8a49e None admin] RBAC: Authorizing `identity:get_project(project_id=08b4540d439e44adb9398aec4130989a)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:39.336355 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110bfd76-7e01-412c-bf75-306cb5f8a49e None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:39.338056 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 218/444] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:39 2022] GET /identity/v3/projects/08b4540d439e44adb9398aec4130989a => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:39.338876 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bfd76-7e01-412c-bf75-306cb5f8a49e None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:39.389076 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:39.415259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 227/445] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:44:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:44:39.416022 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd57325-4f07-4611-abc0-2b078c5c0994 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:39.440207 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bfd76-7e01-412c-bf75-306cb5f8a49e None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:39.475319 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396c3a5-700f-4d16-8ce9-b9b82a5015f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:39.475944 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396c3a5-700f-4d16-8ce9-b9b82a5015f2 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:39.476220 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396c3a5-700f-4d16-8ce9-b9b82a5015f2 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:39.476486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396c3a5-700f-4d16-8ce9-b9b82a5015f2 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:39.477145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a396c3a5-700f-4d16-8ce9-b9b82a5015f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:39.477443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a396c3a5-700f-4d16-8ce9-b9b82a5015f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:39.478845 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a396c3a5-700f-4d16-8ce9-b9b82a5015f2 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:39.481295 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:39.487514 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 219/446] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:39.488317 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396c3a5-700f-4d16-8ce9-b9b82a5015f2 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:39.518709 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd57325-4f07-4611-abc0-2b078c5c0994 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:39.552318 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c502ef0-eeee-414d-8cc5-32bafdfa5bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:39.552961 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c502ef0-eeee-414d-8cc5-32bafdfa5bb2 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:39.553198 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c502ef0-eeee-414d-8cc5-32bafdfa5bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:39.553467 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c502ef0-eeee-414d-8cc5-32bafdfa5bb2 None admin] PATH_INFO: `/v3/groups/b850e638b5bc42f7a7568111b339d0e1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:39.554603 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c502ef0-eeee-414d-8cc5-32bafdfa5bb2 None admin] RBAC: Authorizing `identity:get_group(group_id=b850e638b5bc42f7a7568111b339d0e1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:39.556050 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c502ef0-eeee-414d-8cc5-32bafdfa5bb2 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:39.557616 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 228/447] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 31 10:44:39 2022] GET /identity/v3/groups/b850e638b5bc42f7a7568111b339d0e1 => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:39.558365 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c502ef0-eeee-414d-8cc5-32bafdfa5bb2 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:39.592558 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396c3a5-700f-4d16-8ce9-b9b82a5015f2 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:39.605396 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:39.627845 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62019552-a00b-460f-8190-91e277824d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:39.628468 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62019552-a00b-460f-8190-91e277824d17 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:39.628749 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62019552-a00b-460f-8190-91e277824d17 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:39.629025 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62019552-a00b-460f-8190-91e277824d17 None admin] PATH_INFO: `/v3/projects/08b4540d439e44adb9398aec4130989a` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:39.630167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62019552-a00b-460f-8190-91e277824d17 None admin] RBAC: Authorizing `identity:get_project(project_id=08b4540d439e44adb9398aec4130989a)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:39.631657 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62019552-a00b-460f-8190-91e277824d17 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:39.633303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 220/448] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:39 2022] GET /identity/v3/projects/08b4540d439e44adb9398aec4130989a => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:39.634237 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62019552-a00b-460f-8190-91e277824d17 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:39.660038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c502ef0-eeee-414d-8cc5-32bafdfa5bb2 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:39.694146 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2b278-562d-479d-b866-42da7244355a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:39.694768 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2b278-562d-479d-b866-42da7244355a None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:39.695033 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2b278-562d-479d-b866-42da7244355a None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:39.695302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2b278-562d-479d-b866-42da7244355a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:39.698014 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:39.739184 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62019552-a00b-460f-8190-91e277824d17 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:39.767320 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda2b278-562d-479d-b866-42da7244355a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:39.768814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda2b278-562d-479d-b866-42da7244355a None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:39.774947 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06603217-5e87-4338-9d82-5624ddf4d44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:39.775549 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06603217-5e87-4338-9d82-5624ddf4d44a None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:39.775826 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06603217-5e87-4338-9d82-5624ddf4d44a None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:39.776148 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06603217-5e87-4338-9d82-5624ddf4d44a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:39.776878 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06603217-5e87-4338-9d82-5624ddf4d44a None admin] RBAC: Adding query filter params (group.id=b850e638b5bc42f7a7568111b339d0e1, role.id=31d4b1dc082c41f7b6f18c338e7b51df, scope.project.id=08b4540d439e44adb9398aec4130989a) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:39.777182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06603217-5e87-4338-9d82-5624ddf4d44a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:39.779112 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06603217-5e87-4338-9d82-5624ddf4d44a None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:39.785549 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 221/449] 10.222.0.16 () {56 vars in 1376 bytes} [Wed Aug 31 10:44:39 2022] GET /identity/v3/role_assignments?group.id=b850e638b5bc42f7a7568111b339d0e1&scope.project.id=08b4540d439e44adb9398aec4130989a&role.id=31d4b1dc082c41f7b6f18c338e7b51df => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:39.786319 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06603217-5e87-4338-9d82-5624ddf4d44a None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:39.840033 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:39.906500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06603217-5e87-4338-9d82-5624ddf4d44a None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:39.912592 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 229/450] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:44:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:44:39.913458 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2b278-562d-479d-b866-42da7244355a None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:39.947858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13075d9a-36fb-4759-92bc-102110c52acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:39.948394 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13075d9a-36fb-4759-92bc-102110c52acb None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:39.948675 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13075d9a-36fb-4759-92bc-102110c52acb None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:39.948943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13075d9a-36fb-4759-92bc-102110c52acb None admin] PATH_INFO: `/v3/groups/b850e638b5bc42f7a7568111b339d0e1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:39.950060 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13075d9a-36fb-4759-92bc-102110c52acb None admin] RBAC: Authorizing `identity:get_group(group_id=b850e638b5bc42f7a7568111b339d0e1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:39.952714 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13075d9a-36fb-4759-92bc-102110c52acb None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:39.955620 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 222/451] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 31 10:44:39 2022] GET /identity/v3/groups/b850e638b5bc42f7a7568111b339d0e1 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:39.958718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:39.961175 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13075d9a-36fb-4759-92bc-102110c52acb None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:40.039242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2b278-562d-479d-b866-42da7244355a None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:40.064001 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13075d9a-36fb-4759-92bc-102110c52acb None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:40.073709 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf86b9-01c2-4aa4-92d2-fc3c4a408935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:40.074335 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf86b9-01c2-4aa4-92d2-fc3c4a408935 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:40.074654 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf86b9-01c2-4aa4-92d2-fc3c4a408935 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:40.074930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf86b9-01c2-4aa4-92d2-fc3c4a408935 None admin] PATH_INFO: `/v3/projects/776925ef060044f49d8dab7d8928f73c` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:40.076800 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf86b9-01c2-4aa4-92d2-fc3c4a408935 None admin] RBAC: Authorizing `identity:get_project(project_id=776925ef060044f49d8dab7d8928f73c)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:40.077509 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:40.078227 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf86b9-01c2-4aa4-92d2-fc3c4a408935 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:40.079921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 230/452] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:44:39 2022] GET /identity/v3/projects/776925ef060044f49d8dab7d8928f73c => generated 344 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:40.085488 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf86b9-01c2-4aa4-92d2-fc3c4a408935 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:40.100474 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38dbe6-c0da-4632-a362-e4861fabb53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:40.101077 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38dbe6-c0da-4632-a362-e4861fabb53c None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:40.101352 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38dbe6-c0da-4632-a362-e4861fabb53c None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:40.101629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38dbe6-c0da-4632-a362-e4861fabb53c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:40.102361 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b38dbe6-c0da-4632-a362-e4861fabb53c None admin] RBAC: Adding query filter params (group.id=b850e638b5bc42f7a7568111b339d0e1, role.id=2278da3c52644a819d210b5f9c20b6c1, scope.project.id=08b4540d439e44adb9398aec4130989a) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:40.102705 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b38dbe6-c0da-4632-a362-e4861fabb53c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:40.104135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b38dbe6-c0da-4632-a362-e4861fabb53c None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:40.112262 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 223/453] 10.222.0.16 () {56 vars in 1376 bytes} [Wed Aug 31 10:44:39 2022] GET /identity/v3/role_assignments?group.id=b850e638b5bc42f7a7568111b339d0e1&scope.project.id=08b4540d439e44adb9398aec4130989a&role.id=2278da3c52644a819d210b5f9c20b6c1 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:40.194585 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf86b9-01c2-4aa4-92d2-fc3c4a408935 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:40.196235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:40.231093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e073540f-47eb-49da-b56d-d28208a73e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:40.231799 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073540f-47eb-49da-b56d-d28208a73e76 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:40.232133 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073540f-47eb-49da-b56d-d28208a73e76 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:40.232703 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073540f-47eb-49da-b56d-d28208a73e76 None admin] PATH_INFO: `/v3/groups/13cb05d6493d46898a644db37bd7a699` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:40.233893 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e073540f-47eb-49da-b56d-d28208a73e76 None admin] RBAC: Authorizing `identity:get_group(group_id=13cb05d6493d46898a644db37bd7a699)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:40.235381 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e073540f-47eb-49da-b56d-d28208a73e76 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:40.237000 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 231/454] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 31 10:44:40 2022] GET /identity/v3/groups/13cb05d6493d46898a644db37bd7a699 => generated 229 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:44:40.242993 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38dbe6-c0da-4632-a362-e4861fabb53c None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:40.350245 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38dbe6-c0da-4632-a362-e4861fabb53c None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:40.384589 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b434e-cc1f-4314-a7af-bc508ca75b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:40.385192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b434e-cc1f-4314-a7af-bc508ca75b30 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:40.385463 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b434e-cc1f-4314-a7af-bc508ca75b30 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:40.385728 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b434e-cc1f-4314-a7af-bc508ca75b30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:40.386469 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b434e-cc1f-4314-a7af-bc508ca75b30 None admin] RBAC: Adding query filter params (group.id=13cb05d6493d46898a644db37bd7a699, role.id=13ca33d694c645d6920930d39dc52a0d, scope.project.id=776925ef060044f49d8dab7d8928f73c) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:40.386771 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b434e-cc1f-4314-a7af-bc508ca75b30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:40.388837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b434e-cc1f-4314-a7af-bc508ca75b30 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:40.394932 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 224/455] 10.222.0.16 () {56 vars in 1376 bytes} [Wed Aug 31 10:44:40 2022] GET /identity/v3/role_assignments?group.id=13cb05d6493d46898a644db37bd7a699&scope.project.id=776925ef060044f49d8dab7d8928f73c&role.id=13ca33d694c645d6920930d39dc52a0d => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:42.919974 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d30e947-8544-4362-b111-f33857711650 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:42.920491 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d30e947-8544-4362-b111-f33857711650 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:42.920564 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d30e947-8544-4362-b111-f33857711650 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:42.922647 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 232/456] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:42.928724 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81370b-d259-496e-b3a6-493878686aa9 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:42.929666 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81370b-d259-496e-b3a6-493878686aa9 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:42.930144 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81370b-d259-496e-b3a6-493878686aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:42.955150 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-2d81370b-d259-496e-b3a6-493878686aa9 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:42.956735 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-2d81370b-d259-496e-b3a6-493878686aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:42.996800 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:43.038676 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 225/457] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:43.130956 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a9e29-038e-45e3-a273-75368d3c484e None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:43.131267 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a9e29-038e-45e3-a273-75368d3c484e None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:43.131500 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a9e29-038e-45e3-a273-75368d3c484e None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:43.133101 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 233/458] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:43.139025 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca3941-1299-42b6-b4b8-5a8420e9bdf8 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:43.139307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca3941-1299-42b6-b4b8-5a8420e9bdf8 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:43.139580 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca3941-1299-42b6-b4b8-5a8420e9bdf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:43.158637 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-a4ca3941-1299-42b6-b4b8-5a8420e9bdf8 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:43.159829 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a4ca3941-1299-42b6-b4b8-5a8420e9bdf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:43.172106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4862f-59d8-459a-93a6-04a06a4162b1 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:43.172316 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4862f-59d8-459a-93a6-04a06a4162b1 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:43.172585 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4862f-59d8-459a-93a6-04a06a4162b1 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:43.174191 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 234/459] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:43.179392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffaa626-d08a-4de5-93dc-3c31b3c1367f None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:43.179678 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffaa626-d08a-4de5-93dc-3c31b3c1367f None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:43.179953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffaa626-d08a-4de5-93dc-3c31b3c1367f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:43.210904 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-6ffaa626-d08a-4de5-93dc-3c31b3c1367f None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:43.212113 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-6ffaa626-d08a-4de5-93dc-3c31b3c1367f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:43.284235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 226/460] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:43.327321 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 235/461] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:44:43.453957 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca3941-1299-42b6-b4b8-5a8420e9bdf8 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:43.582816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:43.585552 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca3941-1299-42b6-b4b8-5a8420e9bdf8 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:43.596269 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5875e6b-bcc0-4741-8145-825034302bcc None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:43.596549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5875e6b-bcc0-4741-8145-825034302bcc None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:43.596815 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5875e6b-bcc0-4741-8145-825034302bcc None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:43.598358 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 236/462] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:43.603390 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28137ded-65a1-4133-8983-b24b4cfa8942 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:43.604328 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28137ded-65a1-4133-8983-b24b4cfa8942 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:43.604328 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28137ded-65a1-4133-8983-b24b4cfa8942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:43.623025 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-28137ded-65a1-4133-8983-b24b4cfa8942 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:43.624300 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-28137ded-65a1-4133-8983-b24b4cfa8942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:43.624528 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48dd25-4e96-4bb7-9bbf-2397c6ec30b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:43.625173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48dd25-4e96-4bb7-9bbf-2397c6ec30b4 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:43.625449 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48dd25-4e96-4bb7-9bbf-2397c6ec30b4 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:43.625715 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48dd25-4e96-4bb7-9bbf-2397c6ec30b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:43.636607 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d48dd25-4e96-4bb7-9bbf-2397c6ec30b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:43.638177 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d48dd25-4e96-4bb7-9bbf-2397c6ec30b4 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:43.654779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-5d48dd25-4e96-4bb7-9bbf-2397c6ec30b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:43.654779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:43.654779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:43.654779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:43.654779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:43.654779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:43.654779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:43.654779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:43.654779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:43.654779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:43.654779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:43.654779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:43.654779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:44:43.654779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:44:43.654779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:44:43.654779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:44:43.655701 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:43.655701 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:43.655701 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:43.655701 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:43.655701 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:43.655701 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:43.655701 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:43.655701 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:43.655701 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:43.655701 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:43.655701 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:43.655701 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:43.655701 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:43.655701 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:43.655701 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:43.655701 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:43.655701 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:44:43.656543 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:44:43.656543 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:43.656543 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:43.656543 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 227/463] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:44:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:43.662264 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48dd25-4e96-4bb7-9bbf-2397c6ec30b4 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:43.699696 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 237/464] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:44:43.747054 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28137ded-65a1-4133-8983-b24b4cfa8942 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:43.774098 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48dd25-4e96-4bb7-9bbf-2397c6ec30b4 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:43.799063 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:43.810577 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc8a5e-5543-47a6-b5c0-e9c84765ba58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:43.815948 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc8a5e-5543-47a6-b5c0-e9c84765ba58 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:43.815948 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc8a5e-5543-47a6-b5c0-e9c84765ba58 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:43.815948 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc8a5e-5543-47a6-b5c0-e9c84765ba58 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:43.815948 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc8a5e-5543-47a6-b5c0-e9c84765ba58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:43.815948 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc8a5e-5543-47a6-b5c0-e9c84765ba58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:43.815948 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc8a5e-5543-47a6-b5c0-e9c84765ba58 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:43.827829 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 228/465] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:44:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:43.828934 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc8a5e-5543-47a6-b5c0-e9c84765ba58 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:43.846808 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:43.890139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28137ded-65a1-4133-8983-b24b4cfa8942 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:43.929222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34091890-3420-41ab-9ad4-682e4048765c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:43.930682 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34091890-3420-41ab-9ad4-682e4048765c None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:43.931203 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34091890-3420-41ab-9ad4-682e4048765c None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:43.931661 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34091890-3420-41ab-9ad4-682e4048765c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:43.940696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc8a5e-5543-47a6-b5c0-e9c84765ba58 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:43.944127 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34091890-3420-41ab-9ad4-682e4048765c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:43.945770 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34091890-3420-41ab-9ad4-682e4048765c None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:43.963574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-34091890-3420-41ab-9ad4-682e4048765c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:43.963574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:43.963574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:43.963574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:43.963574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:43.963574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:43.963574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:43.963574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:43.963574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:43.963574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:43.963574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:43.963574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:43.963574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:44:43.963574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:44:43.963574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:44:43.963574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:44:43.964470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:43.964470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:43.964470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:43.964470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:43.964470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:43.964470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:43.964470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:43.964470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:43.964470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:43.964470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:43.964470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:43.964470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:43.964470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:43.964470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:43.964470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:43.964470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:43.964470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:44:43.965402 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:44:43.965402 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:43.965402 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:43.967973 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 238/466] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:44:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:43.969023 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34091890-3420-41ab-9ad4-682e4048765c None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:43.979929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f666bb-be52-4b29-b45f-a107b2c88fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:43.980685 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f666bb-be52-4b29-b45f-a107b2c88fc6 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:43.981058 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f666bb-be52-4b29-b45f-a107b2c88fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:43.981398 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f666bb-be52-4b29-b45f-a107b2c88fc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:43.993068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f666bb-be52-4b29-b45f-a107b2c88fc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:43.995090 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f666bb-be52-4b29-b45f-a107b2c88fc6 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:44.011295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-f1f666bb-be52-4b29-b45f-a107b2c88fc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:44.011295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:44.011295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:44.011295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:44.011295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:44.011295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:44.011295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:44.011295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:44.011295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:44.011295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:44.011295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:44.011295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:44.011295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:44:44.011295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:44:44.011295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:44:44.011295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:44:44.012212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:44.012212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:44.012212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:44.012212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:44.012212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:44.012212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:44.012212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:44.012212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:44.012212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:44.012212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:44.012212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:44.012212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:44.012212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:44.012212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:44.012212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:44.012212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:44.012212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:44:44.013572 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:44:44.013572 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:44.013572 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:44.017549 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 229/467] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:44:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:44.018601 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f666bb-be52-4b29-b45f-a107b2c88fc6 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:44.065103 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:44.086705 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34091890-3420-41ab-9ad4-682e4048765c None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:44.103461 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:44.137254 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbed89-d582-4603-a410-81be09df1452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:44.137863 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbed89-d582-4603-a410-81be09df1452 None admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:44.138140 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbed89-d582-4603-a410-81be09df1452 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:44.138498 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbed89-d582-4603-a410-81be09df1452 None admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:44.139723 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdbed89-d582-4603-a410-81be09df1452 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:44.141384 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdbed89-d582-4603-a410-81be09df1452 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:44.157702 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f666bb-be52-4b29-b45f-a107b2c88fc6 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:44.196470 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0acec5-e32c-4e76-be66-28990d160455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:44.197344 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0acec5-e32c-4e76-be66-28990d160455 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:44.197451 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0acec5-e32c-4e76-be66-28990d160455 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:44.197711 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0acec5-e32c-4e76-be66-28990d160455 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:44.198332 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0acec5-e32c-4e76-be66-28990d160455 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:44.198673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0acec5-e32c-4e76-be66-28990d160455 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:44.200014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0acec5-e32c-4e76-be66-28990d160455 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:44.212771 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 230/468] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:44.213603 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0acec5-e32c-4e76-be66-28990d160455 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:44.241469 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 239/469] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 31 10:44:43 2022] POST /identity/v3/users => generated 246 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:44:44.259535 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbed89-d582-4603-a410-81be09df1452 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:44.354960 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0acec5-e32c-4e76-be66-28990d160455 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:44.393249 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbed89-d582-4603-a410-81be09df1452 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:44.395737 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:44.413642 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f028095-76c0-4e58-b364-7d3e042c9d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:44.414275 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f028095-76c0-4e58-b364-7d3e042c9d76 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:44.414582 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f028095-76c0-4e58-b364-7d3e042c9d76 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:44.415784 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f028095-76c0-4e58-b364-7d3e042c9d76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:44.426379 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f028095-76c0-4e58-b364-7d3e042c9d76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:44.427865 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f028095-76c0-4e58-b364-7d3e042c9d76 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:44.442236 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f250564-6130-4c60-8a14-048a7cdce504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:44.442890 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f250564-6130-4c60-8a14-048a7cdce504 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:44.443172 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f250564-6130-4c60-8a14-048a7cdce504 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:44.443441 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f250564-6130-4c60-8a14-048a7cdce504 None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:44.444097 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f250564-6130-4c60-8a14-048a7cdce504 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:44.444399 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f250564-6130-4c60-8a14-048a7cdce504 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:44.445152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-7f028095-76c0-4e58-b364-7d3e042c9d76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:44.445152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:44.445152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:44.445152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:44.445152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:44.445152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:44.445152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:44.445152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:44.445152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:44.445152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:44.445152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:44.445152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:44.445152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:44:44.445152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:44:44.445152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:44:44.445152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:44:44.447029 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:44.447029 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:44.447029 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:44.447029 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:44.447029 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:44.447029 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:44.447029 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:44.447029 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:44.447029 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:44.447029 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:44.447029 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:44.447029 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:44.447029 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:44.447029 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:44.447029 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:44.447029 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:44.447029 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:44:44.448575 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:44:44.448575 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:44.448575 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:44.448820 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f250564-6130-4c60-8a14-048a7cdce504 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:44.448922 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 231/470] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:44:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:44.448922 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4891d44-13cc-4981-bb11-4ab2c9cac82f None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:44.448922 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4891d44-13cc-4981-bb11-4ab2c9cac82f None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:44.448922 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4891d44-13cc-4981-bb11-4ab2c9cac82f None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:44.449716 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 232/471] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:44.451997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4891d44-13cc-4981-bb11-4ab2c9cac82f None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:44.459437 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:44.465529 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 240/472] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:44.467014 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e5d0a-3f25-4fab-96bd-d3cc0307d85b None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:44.467364 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e5d0a-3f25-4fab-96bd-d3cc0307d85b None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:44.467671 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e5d0a-3f25-4fab-96bd-d3cc0307d85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:44.490453 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-9f4e5d0a-3f25-4fab-96bd-d3cc0307d85b None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:44.490453 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-9f4e5d0a-3f25-4fab-96bd-d3cc0307d85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:44.579472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 241/473] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:44.584476 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e5d0a-3f25-4fab-96bd-d3cc0307d85b None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:44.594871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4891d44-13cc-4981-bb11-4ab2c9cac82f None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:44.639262 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a7272-8c8b-4876-b999-61c74fb7d083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:44.640116 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a7272-8c8b-4876-b999-61c74fb7d083 None admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:44.640969 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a7272-8c8b-4876-b999-61c74fb7d083 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:44.641395 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a7272-8c8b-4876-b999-61c74fb7d083 None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:44.642525 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a7272-8c8b-4876-b999-61c74fb7d083 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:44.644481 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a7272-8c8b-4876-b999-61c74fb7d083 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:44.707186 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e5d0a-3f25-4fab-96bd-d3cc0307d85b None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:44.727548 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 233/474] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 31 10:44:44 2022] POST /identity/v3/users => generated 248 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:44:44.729483 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a7272-8c8b-4876-b999-61c74fb7d083 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:44.746509 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:44.765712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979ac15-147c-4802-b78f-2e7ad72ef6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:44.766510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979ac15-147c-4802-b78f-2e7ad72ef6a3 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:44.766908 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979ac15-147c-4802-b78f-2e7ad72ef6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:44.767250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979ac15-147c-4802-b78f-2e7ad72ef6a3 None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:44.767991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979ac15-147c-4802-b78f-2e7ad72ef6a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:44.768645 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979ac15-147c-4802-b78f-2e7ad72ef6a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:44.769935 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979ac15-147c-4802-b78f-2e7ad72ef6a3 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:44.780588 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:44.794809 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 242/475] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:44.800404 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979ac15-147c-4802-b78f-2e7ad72ef6a3 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:44.911606 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a7272-8c8b-4876-b999-61c74fb7d083 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:44.961977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1046c6-8925-498e-9dc4-4b0796b70922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:44.963107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1046c6-8925-498e-9dc4-4b0796b70922 None admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:44.963107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1046c6-8925-498e-9dc4-4b0796b70922 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:44.963302 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1046c6-8925-498e-9dc4-4b0796b70922 None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:44.964287 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1046c6-8925-498e-9dc4-4b0796b70922 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:44.965559 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1046c6-8925-498e-9dc4-4b0796b70922 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:44.972897 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979ac15-147c-4802-b78f-2e7ad72ef6a3 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:45.033098 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcefd64-3e84-4962-886a-18511a4c3dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:45.034976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcefd64-3e84-4962-886a-18511a4c3dd3 None admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:45.034976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcefd64-3e84-4962-886a-18511a4c3dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:45.034976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcefd64-3e84-4962-886a-18511a4c3dd3 None admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:45.035222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcefd64-3e84-4962-886a-18511a4c3dd3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:45.036519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcefd64-3e84-4962-886a-18511a4c3dd3 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:45.076032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 234/476] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 31 10:44:44 2022] POST /identity/v3/users => generated 248 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:44:45.077108 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1046c6-8925-498e-9dc4-4b0796b70922 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:45.125239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 243/477] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 31 10:44:44 2022] POST /identity/v3/users => generated 249 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 10:44:45.222948 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:45.246915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1046c6-8925-498e-9dc4-4b0796b70922 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:45.301218 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749afcbe-a39c-4b25-b76c-a7fe9bc71048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:45.301846 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749afcbe-a39c-4b25-b76c-a7fe9bc71048 None admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:45.302126 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749afcbe-a39c-4b25-b76c-a7fe9bc71048 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:45.302446 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749afcbe-a39c-4b25-b76c-a7fe9bc71048 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:45.303220 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749afcbe-a39c-4b25-b76c-a7fe9bc71048 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:45.304638 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749afcbe-a39c-4b25-b76c-a7fe9bc71048 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:45.363314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.sql.core [None req-749afcbe-a39c-4b25-b76c-a7fe9bc71048 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 10:44:45.363314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 10:44:45.363314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [parameters: {'id': '873ff54315cb45209694f9d0e92e3067', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 10:44:45.363314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=82089) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 10:44:45.380654 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-749afcbe-a39c-4b25-b76c-a7fe9bc71048 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 10:44:45.380654 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:45.380654 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 10:44:45.380654 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 10:44:45.380654 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 10:44:45.380654 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 10:44:45.380654 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 10:44:45.380654 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 10:44:45.380654 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 10:44:45.380654 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application conn.query(q) Aug 31 10:44:45.380654 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 10:44:45.380654 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 10:44:45.380654 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 10:44:45.380654 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application result.read() Aug 31 10:44:45.380654 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 10:44:45.381670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 10:44:45.381670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 10:44:45.381670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 10:44:45.381670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 10:44:45.381670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 10:44:45.381670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 10:44:45.381670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 10:44:45.381670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 10:44:45.381670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:45.381670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 10:44:45.381670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:45.381670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:45.381670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 10:44:45.381670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 10:44:45.381670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 31 10:44:45.381670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return ref.to_dict() Aug 31 10:44:45.381670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 10:44:45.381670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application next(self.gen) Aug 31 10:44:45.382566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 10:44:45.382566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application yield resource Aug 31 10:44:45.382566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 10:44:45.382566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application next(self.gen) Aug 31 10:44:45.382566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 10:44:45.382566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 10:44:45.382566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 10:44:45.382566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 10:44:45.382566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 10:44:45.382566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise self.value Aug 31 10:44:45.382566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 10:44:45.382566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 10:44:45.382566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 10:44:45.382566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application session.commit() Aug 31 10:44:45.382566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 10:44:45.382566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 10:44:45.382566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 10:44:45.383429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 10:44:45.383429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 10:44:45.383429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self.session.flush() Aug 31 10:44:45.383429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 10:44:45.383429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 10:44:45.383429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 10:44:45.383429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 10:44:45.383429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 10:44:45.383429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application compat.raise_( Aug 31 10:44:45.383429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 10:44:45.383429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception Aug 31 10:44:45.383429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 10:44:45.383429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 10:44:45.383429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 10:44:45.383429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 10:44:45.383429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 10:44:45.383429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 10:44:45.384338 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 10:44:45.384338 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 10:44:45.384338 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 10:44:45.384338 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 10:44:45.384338 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 10:44:45.384338 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 10:44:45.384338 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 10:44:45.384338 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 10:44:45.384338 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 10:44:45.384338 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 10:44:45.384338 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 10:44:45.384338 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 10:44:45.384338 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 10:44:45.384338 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 10:44:45.384338 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 10:44:45.384338 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception Aug 31 10:44:45.385151 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 10:44:45.385151 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 10:44:45.385151 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 10:44:45.385151 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 10:44:45.385151 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 10:44:45.385151 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 10:44:45.385151 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 10:44:45.385151 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application conn.query(q) Aug 31 10:44:45.385151 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 10:44:45.385151 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 10:44:45.385151 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 10:44:45.385151 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application result.read() Aug 31 10:44:45.385151 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 10:44:45.385151 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 10:44:45.385151 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 10:44:45.385151 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 10:44:45.385898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 10:44:45.385898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 10:44:45.385898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 10:44:45.385898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 10:44:45.385898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 10:44:45.385898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 10:44:45.385898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [parameters: {'id': '873ff54315cb45209694f9d0e92e3067', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 10:44:45.385898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 10:44:45.385898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:45.385898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:44:45.385898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:45.385898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:45.385898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:45.385898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:45.385898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:45.385898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:45.388795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:45.388795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:45.388795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:45.388795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:45.388795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:45.388795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:45.388795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 31 10:44:45.388795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 31 10:44:45.388795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:45.388795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:45.388795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 31 10:44:45.388795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 31 10:44:45.388795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 10:44:45.388795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 10:44:45.388795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 10:44:45.388795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:45.391018 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 235/478] 10.222.0.16 () {60 vars in 1131 bytes} [Wed Aug 31 10:44:45 2022] POST /identity/v3/roles => generated 144 bytes in 305 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:45.391150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcefd64-3e84-4962-886a-18511a4c3dd3 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:45.451443 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:45.506522 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcefd64-3e84-4962-886a-18511a4c3dd3 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:45.554067 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed47b3-ed65-49f9-b7a2-bb5ab952d1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:45.554766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed47b3-ed65-49f9-b7a2-bb5ab952d1ae None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:45.555050 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed47b3-ed65-49f9-b7a2-bb5ab952d1ae None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:45.555321 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed47b3-ed65-49f9-b7a2-bb5ab952d1ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:45.572861 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed47b3-ed65-49f9-b7a2-bb5ab952d1ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:45.579562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-ffed47b3-ed65-49f9-b7a2-bb5ab952d1ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:44:45.579562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:45.579562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:45.579562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:45.579562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:45.579562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:45.579562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:45.579562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:45.579562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:45.579562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:45.579562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:45.579562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:45.579562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:45.579562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:45.579562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:45.579562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:45.580486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:45.580486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:45.580486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:45.580486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:45.580486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:45.580486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:45.580486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:45.580486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:45.580486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:45.580486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:45.580486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:45.580486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:45.580486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:45.580486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:45.580486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:45.580486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:45.581285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:45.581285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:45.581285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:45.581285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:45.581285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:45.581285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:45.581285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:45.581285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:45.581285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:45.581285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:45.581285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:44:45.581285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:45.585697 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 244/479] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 31 10:44:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:45.587106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749afcbe-a39c-4b25-b76c-a7fe9bc71048 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:45.703282 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749afcbe-a39c-4b25-b76c-a7fe9bc71048 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:45.746344 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968cd1c2-dba9-4cf2-9d4a-8fa88648bda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:45.746983 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cd1c2-dba9-4cf2-9d4a-8fa88648bda2 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:45.747802 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cd1c2-dba9-4cf2-9d4a-8fa88648bda2 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:45.748075 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cd1c2-dba9-4cf2-9d4a-8fa88648bda2 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:45.748866 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968cd1c2-dba9-4cf2-9d4a-8fa88648bda2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:45.749165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968cd1c2-dba9-4cf2-9d4a-8fa88648bda2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:45.750573 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968cd1c2-dba9-4cf2-9d4a-8fa88648bda2 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:45.767225 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 236/480] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:44:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:46.660456 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e52300-bfbe-48b0-80b4-1ac9434cd1f8 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:46.661496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e52300-bfbe-48b0-80b4-1ac9434cd1f8 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:46.661981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e52300-bfbe-48b0-80b4-1ac9434cd1f8 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:46.663879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 245/481] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:46.671064 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f733c4-1eff-4011-aae6-aa7a620aaa51 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:46.671998 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f733c4-1eff-4011-aae6-aa7a620aaa51 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:46.672474 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f733c4-1eff-4011-aae6-aa7a620aaa51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:46.707689 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-f5f733c4-1eff-4011-aae6-aa7a620aaa51 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:46.709273 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-f5f733c4-1eff-4011-aae6-aa7a620aaa51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:46.721406 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:46.799192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 237/482] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:44:47.225932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a71f3-0887-4d6d-abec-efaa1b0dadf1 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:47.226303 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a71f3-0887-4d6d-abec-efaa1b0dadf1 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:47.226535 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a71f3-0887-4d6d-abec-efaa1b0dadf1 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:47.228100 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 246/483] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:47.229378 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f733c4-1eff-4011-aae6-aa7a620aaa51 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:47.250149 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb3ed9-6867-4207-a156-dd6484101264 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:47.250712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb3ed9-6867-4207-a156-dd6484101264 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:47.251167 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb3ed9-6867-4207-a156-dd6484101264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:47.283518 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-b3bb3ed9-6867-4207-a156-dd6484101264 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:47.285051 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-b3bb3ed9-6867-4207-a156-dd6484101264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:47.353703 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f733c4-1eff-4011-aae6-aa7a620aaa51 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:47.413748 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 247/484] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:44:47.422353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc901d9-425c-4068-9824-0ff377afe4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:47.423579 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc901d9-425c-4068-9824-0ff377afe4c1 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:47.424207 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc901d9-425c-4068-9824-0ff377afe4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:47.424827 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc901d9-425c-4068-9824-0ff377afe4c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:47.437406 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc901d9-425c-4068-9824-0ff377afe4c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:47.442590 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-ecc901d9-425c-4068-9824-0ff377afe4c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:44:47.442590 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:47.442590 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:47.442590 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:47.442590 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:47.442590 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:47.442590 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:47.442590 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:47.442590 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:47.442590 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:47.442590 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:47.442590 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:47.442590 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:47.442590 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:47.442590 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:47.442590 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:47.443738 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:47.443738 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:47.443738 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:47.443738 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:47.443738 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:47.443738 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:47.443738 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:47.443738 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:47.443738 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:47.443738 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:47.443738 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:47.443738 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:47.443738 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:47.443738 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:47.443738 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:47.443738 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:47.444705 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:47.444705 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:47.444705 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:47.444705 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:47.444705 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:47.444705 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:47.444705 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:47.444705 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:47.444705 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:47.444705 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:47.444705 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:44:47.444705 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:47.446754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 238/485] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 31 10:44:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:47.454088 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb3ed9-6867-4207-a156-dd6484101264 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:47.497794 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:47.538054 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e7de5-8738-4cae-9b00-849cdc1a395c None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:47.538335 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e7de5-8738-4cae-9b00-849cdc1a395c None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:47.538783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e7de5-8738-4cae-9b00-849cdc1a395c None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:47.540607 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 239/486] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:47.547310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d848873-43ef-4e62-8f63-b5b56ffa6649 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:47.547787 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d848873-43ef-4e62-8f63-b5b56ffa6649 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:47.548189 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d848873-43ef-4e62-8f63-b5b56ffa6649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:47.568635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:47.590655 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-7d848873-43ef-4e62-8f63-b5b56ffa6649 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:47.594270 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-7d848873-43ef-4e62-8f63-b5b56ffa6649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:47.594382 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb3ed9-6867-4207-a156-dd6484101264 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:47.647795 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452fce6-e3b9-42ea-8f62-2e3bce1017f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:47.648423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452fce6-e3b9-42ea-8f62-2e3bce1017f5 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:47.648700 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452fce6-e3b9-42ea-8f62-2e3bce1017f5 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:47.648972 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452fce6-e3b9-42ea-8f62-2e3bce1017f5 None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:47.650436 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b452fce6-e3b9-42ea-8f62-2e3bce1017f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:47.650725 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b452fce6-e3b9-42ea-8f62-2e3bce1017f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:47.652140 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b452fce6-e3b9-42ea-8f62-2e3bce1017f5 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:47.680670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 248/487] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 31 10:44:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:47.686503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452fce6-e3b9-42ea-8f62-2e3bce1017f5 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:47.699158 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 240/488] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:47.789854 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d848873-43ef-4e62-8f63-b5b56ffa6649 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:47.871203 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:47.879999 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452fce6-e3b9-42ea-8f62-2e3bce1017f5 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:47.925553 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d848873-43ef-4e62-8f63-b5b56ffa6649 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:47.933173 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b808ea51-e71e-416e-bb27-214b15ebefb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:47.933882 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808ea51-e71e-416e-bb27-214b15ebefb0 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:47.934300 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808ea51-e71e-416e-bb27-214b15ebefb0 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:47.935695 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808ea51-e71e-416e-bb27-214b15ebefb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:47.949365 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b808ea51-e71e-416e-bb27-214b15ebefb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:47.951488 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b808ea51-e71e-416e-bb27-214b15ebefb0 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:47.970187 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf840174-65a5-49b3-b475-89d3bd1307b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:47.970864 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf840174-65a5-49b3-b475-89d3bd1307b8 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:47.971138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf840174-65a5-49b3-b475-89d3bd1307b8 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:47.971411 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf840174-65a5-49b3-b475-89d3bd1307b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:47.971510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-b808ea51-e71e-416e-bb27-214b15ebefb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:47.971510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:47.971510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:47.971510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:47.971510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:47.971510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:47.971510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:47.971510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:47.971510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:47.971510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:47.971510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:47.971510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:47.971510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:44:47.971510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:44:47.971510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:44:47.971510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:44:47.972318 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:47.972318 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:47.972318 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:47.972318 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:47.972318 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:47.972318 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:47.972318 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:47.972318 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:47.972318 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:47.972318 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:47.972318 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:47.972318 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:47.972318 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:47.972318 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:47.972318 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:47.972318 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:47.972318 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:44:47.973157 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:44:47.973157 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:47.973157 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:47.974473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 249/489] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:44:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:44:47.975784 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2b382-9849-41e5-8230-3c6789e69eb5 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:47.976124 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2b382-9849-41e5-8230-3c6789e69eb5 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:47.976422 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2b382-9849-41e5-8230-3c6789e69eb5 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:47.977844 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:47.978673 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 250/490] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:47.980867 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a2b382-9849-41e5-8230-3c6789e69eb5 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:47.991041 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf840174-65a5-49b3-b475-89d3bd1307b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:47.997598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-bf840174-65a5-49b3-b475-89d3bd1307b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:44:47.997598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:47.997598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:47.997598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:47.997598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:47.997598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:47.997598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:47.997598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:47.997598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:47.997598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:47.997598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:47.997598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:47.997598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:47.997598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:47.997598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:47.997598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:47.998570 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:47.998570 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:47.998570 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:47.998570 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:47.998570 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:47.998570 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:47.998570 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:47.998570 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:47.998570 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:47.998570 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:47.998570 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:47.998570 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:47.998570 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:47.998570 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:47.998570 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:47.998570 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:47.999670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:47.999670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:47.999670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:47.999670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:47.999670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:47.999670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:47.999670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:47.999670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:47.999670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:47.999670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:47.999670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:44:47.999670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:47.999670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 241/491] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 31 10:44:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:48.004361 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ade2b-ccef-4165-bf8c-a5f8d83d3bd3 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:48.005733 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ade2b-ccef-4165-bf8c-a5f8d83d3bd3 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:48.006010 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ade2b-ccef-4165-bf8c-a5f8d83d3bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:48.025824 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-d61ade2b-ccef-4165-bf8c-a5f8d83d3bd3 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:48.027160 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-d61ade2b-ccef-4165-bf8c-a5f8d83d3bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:48.110686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 242/492] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:48.111757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61ade2b-ccef-4165-bf8c-a5f8d83d3bd3 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:48.131261 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a2b382-9849-41e5-8230-3c6789e69eb5 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:48.182510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f07ea-c483-4ea8-afd6-50313168cfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:48.183946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f07ea-c483-4ea8-afd6-50313168cfeb None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:48.184672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f07ea-c483-4ea8-afd6-50313168cfeb None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:48.185145 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f07ea-c483-4ea8-afd6-50313168cfeb None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:48.186111 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f07ea-c483-4ea8-afd6-50313168cfeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:48.186645 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f07ea-c483-4ea8-afd6-50313168cfeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:48.188535 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f07ea-c483-4ea8-afd6-50313168cfeb None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:48.205095 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 251/493] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:44:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:48.206794 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f07ea-c483-4ea8-afd6-50313168cfeb None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:48.225047 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:48.229199 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61ade2b-ccef-4165-bf8c-a5f8d83d3bd3 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:48.254521 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:48.271295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe7f85-c2ec-4cae-9fc3-d6d002d4ab93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:48.272672 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe7f85-c2ec-4cae-9fc3-d6d002d4ab93 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:48.273147 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe7f85-c2ec-4cae-9fc3-d6d002d4ab93 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:48.273609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe7f85-c2ec-4cae-9fc3-d6d002d4ab93 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:48.274498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe7f85-c2ec-4cae-9fc3-d6d002d4ab93 None admin] RBAC: Adding query filter params (name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:48.274997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe7f85-c2ec-4cae-9fc3-d6d002d4ab93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:48.277310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe7f85-c2ec-4cae-9fc3-d6d002d4ab93 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:48.290611 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 243/494] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 31 10:44:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:48.291800 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe7f85-c2ec-4cae-9fc3-d6d002d4ab93 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:48.340676 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f07ea-c483-4ea8-afd6-50313168cfeb None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:48.379656 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dde0f9-0da6-4979-861c-bc773ea08cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:48.380620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dde0f9-0da6-4979-861c-bc773ea08cdd None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:48.381073 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dde0f9-0da6-4979-861c-bc773ea08cdd None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:48.381516 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dde0f9-0da6-4979-861c-bc773ea08cdd None admin] PATH_INFO: `/v3/roles/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:48.394149 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dde0f9-0da6-4979-861c-bc773ea08cdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:48.399392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-67dde0f9-0da6-4979-861c-bc773ea08cdd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:44:48.399392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:48.399392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:48.399392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:48.399392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:48.399392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:48.399392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:48.399392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:48.399392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:48.399392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:48.399392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:48.399392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:48.399392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:48.399392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:48.399392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:48.399392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:48.400258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:48.400258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:48.400258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:48.400258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:48.400258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:48.400258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:48.400258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:48.400258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:48.400258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:48.400258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:48.400258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:48.400258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:48.400258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:48.400258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:48.400258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:48.400258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:48.401042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:48.401042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:48.401042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:48.401042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:48.401042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:48.401042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:48.401042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:48.401042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:48.401042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:48.401042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:48.401042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:44:48.401042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:48.409415 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 252/495] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 31 10:44:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:48.410678 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dde0f9-0da6-4979-861c-bc773ea08cdd None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:48.434102 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe7f85-c2ec-4cae-9fc3-d6d002d4ab93 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:48.473477 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3a09f-3100-495a-8257-d6f7ab89c543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:48.474105 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3a09f-3100-495a-8257-d6f7ab89c543 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:48.474427 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3a09f-3100-495a-8257-d6f7ab89c543 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:48.474696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3a09f-3100-495a-8257-d6f7ab89c543 None admin] PATH_INFO: `/v3/users/nova` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:48.482486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e3a09f-3100-495a-8257-d6f7ab89c543 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:48.483955 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e3a09f-3100-495a-8257-d6f7ab89c543 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:48.498694 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:48.499782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-51e3a09f-3100-495a-8257-d6f7ab89c543 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:44:48.499782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:48.499782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:48.499782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:48.499782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:48.499782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:48.499782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:48.499782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:48.499782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:48.499782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:48.499782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:48.499782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:48.499782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:44:48.499782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:44:48.499782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:44:48.499782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:44:48.501095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:48.501095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:48.501095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:44:48.501095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:44:48.501095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:44:48.501095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:44:48.501095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:48.501095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:48.501095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:48.501095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:48.501095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:48.501095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:48.501095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:48.501095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:48.501095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:48.501095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:48.501095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:48.501095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:48.502658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:48.502658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:48.502658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:44:48.502658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:44:48.502658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:44:48.502658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:44:48.502658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:44:48.502658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:44:48.502658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:44:48.502658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:48.502658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 244/496] 10.222.0.16 () {56 vars in 1093 bytes} [Wed Aug 31 10:44:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:48.502658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3a09f-3100-495a-8257-d6f7ab89c543 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:48.542883 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dde0f9-0da6-4979-861c-bc773ea08cdd None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:48.556462 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:48.580478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723e0fd-1b8a-431b-ac2c-5ff5d72d75cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:48.581635 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723e0fd-1b8a-431b-ac2c-5ff5d72d75cb None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:48.582120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723e0fd-1b8a-431b-ac2c-5ff5d72d75cb None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:48.583125 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723e0fd-1b8a-431b-ac2c-5ff5d72d75cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:48.597470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0723e0fd-1b8a-431b-ac2c-5ff5d72d75cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:48.598992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0723e0fd-1b8a-431b-ac2c-5ff5d72d75cb None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:48.615670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-0723e0fd-1b8a-431b-ac2c-5ff5d72d75cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:48.615670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:48.615670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:48.615670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:48.615670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:48.615670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:48.615670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:48.615670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:48.615670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:48.615670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:48.615670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:48.615670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:48.615670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:44:48.615670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:44:48.615670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:44:48.615670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:44:48.617253 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:48.617253 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:48.617253 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:48.617253 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:48.617253 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:48.617253 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:48.617253 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:48.617253 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:48.617253 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:48.617253 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:48.617253 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:48.617253 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:48.617253 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:48.617253 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:48.617253 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:48.617253 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:48.617253 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:44:48.618726 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:44:48.618726 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:48.618726 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:48.618726 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 253/497] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:44:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:48.618726 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723e0fd-1b8a-431b-ac2c-5ff5d72d75cb None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:48.624106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3a09f-3100-495a-8257-d6f7ab89c543 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:48.663768 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07581026-65ab-46a0-8cbe-8bd62d456968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:48.664379 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07581026-65ab-46a0-8cbe-8bd62d456968 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:48.664653 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07581026-65ab-46a0-8cbe-8bd62d456968 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:48.664927 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07581026-65ab-46a0-8cbe-8bd62d456968 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:48.665589 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07581026-65ab-46a0-8cbe-8bd62d456968 None admin] RBAC: Adding query filter params (name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:48.665885 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07581026-65ab-46a0-8cbe-8bd62d456968 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:48.667298 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07581026-65ab-46a0-8cbe-8bd62d456968 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:48.676868 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 245/498] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 31 10:44:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:48.677698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07581026-65ab-46a0-8cbe-8bd62d456968 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:48.736823 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723e0fd-1b8a-431b-ac2c-5ff5d72d75cb None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:48.757116 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:48.773099 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a01db-e1b9-4a16-8680-963e38d45670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:48.773772 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a01db-e1b9-4a16-8680-963e38d45670 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:48.774044 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a01db-e1b9-4a16-8680-963e38d45670 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:48.774315 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a01db-e1b9-4a16-8680-963e38d45670 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:48.791075 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6a01db-e1b9-4a16-8680-963e38d45670 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:48.797761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07581026-65ab-46a0-8cbe-8bd62d456968 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:48.798250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-db6a01db-e1b9-4a16-8680-963e38d45670 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:44:48.798250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:48.798250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:48.798250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:48.798250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:48.798250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:48.798250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:48.798250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:48.798250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:48.798250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:48.798250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:48.798250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:48.798250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:48.798250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:48.798250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:48.798250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:48.799211 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:48.799211 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:48.799211 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:48.799211 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:48.799211 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:48.799211 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:48.799211 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:48.799211 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:48.799211 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:48.799211 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:48.799211 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:48.799211 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:48.799211 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:48.799211 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:48.799211 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:48.799211 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:48.799981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:48.799981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:48.799981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:48.799981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:48.799981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:48.799981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:48.799981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:48.799981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:48.799981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:48.799981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:48.799981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:44:48.799981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:48.799981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 254/499] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 31 10:44:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:48.800633 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7196830-db41-4ec6-980d-c712d912d0f0 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:48.800633 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7196830-db41-4ec6-980d-c712d912d0f0 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:48.800633 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7196830-db41-4ec6-980d-c712d912d0f0 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:48.801783 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 255/500] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:48.803063 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7196830-db41-4ec6-980d-c712d912d0f0 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:48.812237 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:48.839432 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dab0af-5f54-4bee-8418-220677521b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:48.840114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dab0af-5f54-4bee-8418-220677521b09 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:48.840395 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dab0af-5f54-4bee-8418-220677521b09 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:48.841233 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dab0af-5f54-4bee-8418-220677521b09 None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:48.842067 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dab0af-5f54-4bee-8418-220677521b09 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:48.842665 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dab0af-5f54-4bee-8418-220677521b09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:48.844703 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dab0af-5f54-4bee-8418-220677521b09 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:48.883893 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 246/501] 10.222.0.16 () {56 vars in 1133 bytes} [Wed Aug 31 10:44:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:48.884838 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dab0af-5f54-4bee-8418-220677521b09 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:48.926414 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7196830-db41-4ec6-980d-c712d912d0f0 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:48.962682 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a94b35-270e-4d8c-95cb-a8b87493fab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:48.963295 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a94b35-270e-4d8c-95cb-a8b87493fab2 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:48.963570 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a94b35-270e-4d8c-95cb-a8b87493fab2 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:48.963841 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a94b35-270e-4d8c-95cb-a8b87493fab2 None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:48.964485 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a94b35-270e-4d8c-95cb-a8b87493fab2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:48.964786 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a94b35-270e-4d8c-95cb-a8b87493fab2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:48.966947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a94b35-270e-4d8c-95cb-a8b87493fab2 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:48.978510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 256/502] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:44:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:48.979270 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a94b35-270e-4d8c-95cb-a8b87493fab2 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:48.994421 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dab0af-5f54-4bee-8418-220677521b09 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:49.024109 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:49.035615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c0a124-01ac-4346-b9b0-d99ed4e93743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:49.036261 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0a124-01ac-4346-b9b0-d99ed4e93743 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:49.036540 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0a124-01ac-4346-b9b0-d99ed4e93743 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:49.041576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0a124-01ac-4346-b9b0-d99ed4e93743 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:49.048533 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c0a124-01ac-4346-b9b0-d99ed4e93743 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:49.050006 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c0a124-01ac-4346-b9b0-d99ed4e93743 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:49.066350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-a1c0a124-01ac-4346-b9b0-d99ed4e93743 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:49.066350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:49.066350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:49.066350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:49.066350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:49.066350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:49.066350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:49.066350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:49.066350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:49.066350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:49.066350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:49.066350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:49.066350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:44:49.066350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:44:49.066350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:44:49.066350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:44:49.068662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:49.068662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:49.068662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:49.068662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:49.068662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:49.068662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:49.068662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:49.068662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:49.068662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:49.068662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:49.068662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:49.068662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:49.068662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:49.068662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:49.068662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:49.068662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:49.068662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:44:49.069699 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:44:49.069699 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:49.069699 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:49.069699 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 247/503] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:44:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:49.069699 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c661d3-9fe7-45b3-a0d2-abd25e5cadbc None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:49.069699 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c661d3-9fe7-45b3-a0d2-abd25e5cadbc None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:49.069699 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c661d3-9fe7-45b3-a0d2-abd25e5cadbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:49.087970 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-a1c661d3-9fe7-45b3-a0d2-abd25e5cadbc None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:49.089177 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a1c661d3-9fe7-45b3-a0d2-abd25e5cadbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:49.092386 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a94b35-270e-4d8c-95cb-a8b87493fab2 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:49.103596 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:49.135010 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda5800-d277-4a54-95fc-6f17b2db3c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:49.135613 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda5800-d277-4a54-95fc-6f17b2db3c8d None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:49.135895 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda5800-d277-4a54-95fc-6f17b2db3c8d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:49.136160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda5800-d277-4a54-95fc-6f17b2db3c8d None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:49.136832 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cda5800-d277-4a54-95fc-6f17b2db3c8d None admin] RBAC: Adding query filter params (name=service) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:49.137134 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cda5800-d277-4a54-95fc-6f17b2db3c8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:49.138549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cda5800-d277-4a54-95fc-6f17b2db3c8d None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:49.146976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 257/504] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 31 10:44:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:49.147807 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda5800-d277-4a54-95fc-6f17b2db3c8d None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:49.180004 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 248/505] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:49.180867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c661d3-9fe7-45b3-a0d2-abd25e5cadbc None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:49.262128 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda5800-d277-4a54-95fc-6f17b2db3c8d None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:49.264243 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:49.314838 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5faeede-568a-438c-8dfc-4b98f2a159eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:49.315550 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5faeede-568a-438c-8dfc-4b98f2a159eb None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:49.315974 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5faeede-568a-438c-8dfc-4b98f2a159eb None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:49.316317 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5faeede-568a-438c-8dfc-4b98f2a159eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:49.319399 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c661d3-9fe7-45b3-a0d2-abd25e5cadbc None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:49.362214 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93a193-12c6-475a-b4e8-291879c9aba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:49.362892 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93a193-12c6-475a-b4e8-291879c9aba1 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:49.363201 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93a193-12c6-475a-b4e8-291879c9aba1 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:49.363935 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93a193-12c6-475a-b4e8-291879c9aba1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:49.373563 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e93a193-12c6-475a-b4e8-291879c9aba1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:49.375088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e93a193-12c6-475a-b4e8-291879c9aba1 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:49.387179 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-8e93a193-12c6-475a-b4e8-291879c9aba1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 10:44:49.387179 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:49.387179 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:49.387179 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:49.387179 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:49.387179 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:49.387179 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:49.387179 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:49.387179 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:49.387179 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:49.387179 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:49.387179 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:49.387179 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:44:49.387179 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:44:49.387179 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:44:49.387179 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:44:49.388754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:49.388754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:49.388754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:44:49.388754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:44:49.388754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:44:49.388754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:44:49.388754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:49.388754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:49.388754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:49.388754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:49.388754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:49.388754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:49.388754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:49.388754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:49.388754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:49.388754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:49.388754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:49.388754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:49.389851 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:49.389851 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:49.389851 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:44:49.389851 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:44:49.389851 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:44:49.389851 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:44:49.389851 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:44:49.389851 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:44:49.389851 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 10:44:49.389851 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:49.393346 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 249/506] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:49.394262 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93a193-12c6-475a-b4e8-291879c9aba1 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:49.406612 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5faeede-568a-438c-8dfc-4b98f2a159eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:49.408369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5faeede-568a-438c-8dfc-4b98f2a159eb None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:49.413306 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:49.514024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93a193-12c6-475a-b4e8-291879c9aba1 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:49.515327 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 258/507] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:44:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:44:49.516253 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5faeede-568a-438c-8dfc-4b98f2a159eb None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:49.518707 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:49.549965 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb72d20-a3ac-4075-8d2e-5a79f85dccbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:49.550652 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb72d20-a3ac-4075-8d2e-5a79f85dccbb None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:49.550925 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb72d20-a3ac-4075-8d2e-5a79f85dccbb None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:49.551191 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb72d20-a3ac-4075-8d2e-5a79f85dccbb None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:49.551829 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb72d20-a3ac-4075-8d2e-5a79f85dccbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:49.552129 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb72d20-a3ac-4075-8d2e-5a79f85dccbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:49.553509 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb72d20-a3ac-4075-8d2e-5a79f85dccbb None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:49.570818 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 250/508] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:44:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:49.570818 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb72d20-a3ac-4075-8d2e-5a79f85dccbb None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:49.635816 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5faeede-568a-438c-8dfc-4b98f2a159eb None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:49.641901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:49.670821 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172984c9-d4a1-48d4-b02d-66c4c48b94e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:49.671448 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172984c9-d4a1-48d4-b02d-66c4c48b94e4 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:49.671720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172984c9-d4a1-48d4-b02d-66c4c48b94e4 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:49.672000 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172984c9-d4a1-48d4-b02d-66c4c48b94e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:49.682715 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172984c9-d4a1-48d4-b02d-66c4c48b94e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:49.684109 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb72d20-a3ac-4075-8d2e-5a79f85dccbb None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:49.684247 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172984c9-d4a1-48d4-b02d-66c4c48b94e4 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:49.698507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-172984c9-d4a1-48d4-b02d-66c4c48b94e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:49.698507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:49.698507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:49.698507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:49.698507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:49.698507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:49.698507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:49.698507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:49.698507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:49.698507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:49.698507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:49.698507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:49.698507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:44:49.698507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:44:49.698507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:44:49.698507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:44:49.699672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:49.699672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:49.699672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:49.699672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:49.699672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:49.699672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:49.699672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:49.699672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:49.699672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:49.699672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:49.699672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:49.699672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:49.699672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:49.699672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:49.699672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:49.699672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:49.699672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:44:49.702045 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:44:49.702045 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:49.702045 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:49.702045 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 259/509] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:44:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:49.702045 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172984c9-d4a1-48d4-b02d-66c4c48b94e4 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:49.727817 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880561e-5afd-4028-9146-f208d88ce013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:49.728434 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880561e-5afd-4028-9146-f208d88ce013 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:49.728860 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880561e-5afd-4028-9146-f208d88ce013 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:49.729131 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880561e-5afd-4028-9146-f208d88ce013 None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:49.729852 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9880561e-5afd-4028-9146-f208d88ce013 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:49.730159 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9880561e-5afd-4028-9146-f208d88ce013 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:49.731898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9880561e-5afd-4028-9146-f208d88ce013 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:49.749866 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 251/510] 10.222.0.16 () {56 vars in 1137 bytes} [Wed Aug 31 10:44:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:49.750747 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880561e-5afd-4028-9146-f208d88ce013 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:49.764343 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:49.812472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172984c9-d4a1-48d4-b02d-66c4c48b94e4 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:49.849931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294bbb1-1eee-4224-902f-0742e34aaa45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:49.850608 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294bbb1-1eee-4224-902f-0742e34aaa45 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:49.850888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294bbb1-1eee-4224-902f-0742e34aaa45 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:49.851158 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294bbb1-1eee-4224-902f-0742e34aaa45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:49.863061 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294bbb1-1eee-4224-902f-0742e34aaa45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:49.864554 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294bbb1-1eee-4224-902f-0742e34aaa45 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:49.865241 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880561e-5afd-4028-9146-f208d88ce013 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:49.883199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-1294bbb1-1eee-4224-902f-0742e34aaa45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:49.883199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:49.883199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:49.883199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:49.883199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:49.883199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:49.883199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:49.883199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:49.883199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:49.883199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:49.883199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:49.883199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:49.883199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:44:49.883199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:44:49.883199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:44:49.883199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:44:49.884238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:49.884238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:49.884238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:49.884238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:49.884238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:49.884238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:49.884238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:49.884238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:49.884238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:49.884238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:49.884238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:49.884238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:49.884238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:49.884238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:49.884238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:49.884238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:49.884238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:44:49.885478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:44:49.885478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:49.885478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:49.885478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 260/511] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:44:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:49.885478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294bbb1-1eee-4224-902f-0742e34aaa45 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:49.906632 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:49.917055 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f13f13-60c5-452f-8e7e-ba968f2ffaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:49.917688 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f13f13-60c5-452f-8e7e-ba968f2ffaae None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:49.917960 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f13f13-60c5-452f-8e7e-ba968f2ffaae None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:49.921993 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f13f13-60c5-452f-8e7e-ba968f2ffaae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:49.929689 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f13f13-60c5-452f-8e7e-ba968f2ffaae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:49.932356 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f13f13-60c5-452f-8e7e-ba968f2ffaae None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:49.947934 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-e1f13f13-60c5-452f-8e7e-ba968f2ffaae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:49.947934 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:49.947934 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:49.947934 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:49.947934 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:49.947934 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:49.947934 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:49.947934 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:49.947934 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:49.947934 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:49.947934 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:49.947934 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:49.947934 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:44:49.947934 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:44:49.947934 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:44:49.947934 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:44:49.949677 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:49.949677 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:49.949677 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:49.949677 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:49.949677 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:49.949677 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:49.949677 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:49.949677 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:49.949677 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:49.949677 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:49.949677 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:49.949677 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:49.949677 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:49.949677 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:49.949677 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:49.949677 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:49.949677 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:44:49.951569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:44:49.951569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:49.951569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:49.951569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 252/512] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:44:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:49.951569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f13f13-60c5-452f-8e7e-ba968f2ffaae None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:50.019296 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294bbb1-1eee-4224-902f-0742e34aaa45 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:50.027367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:50.053892 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f13f13-60c5-452f-8e7e-ba968f2ffaae None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:50.063703 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99068d-8caa-4444-a119-6945bba8b5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:50.064338 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99068d-8caa-4444-a119-6945bba8b5d7 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:50.064615 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99068d-8caa-4444-a119-6945bba8b5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:50.064886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99068d-8caa-4444-a119-6945bba8b5d7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:50.072483 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec99068d-8caa-4444-a119-6945bba8b5d7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:50.073947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec99068d-8caa-4444-a119-6945bba8b5d7 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:50.088946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-ec99068d-8caa-4444-a119-6945bba8b5d7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 10:44:50.088946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:50.088946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:50.088946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:50.088946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:50.088946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:50.088946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:50.088946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:50.088946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:50.088946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:50.088946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:50.088946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:50.088946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:44:50.088946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:44:50.088946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:44:50.088946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:44:50.090129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:50.090129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:50.090129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:44:50.090129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:44:50.090129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:44:50.090129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:44:50.090129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:50.090129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:50.090129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:50.090129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:50.090129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:50.090129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:50.090129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:50.090129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:50.090129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:50.090129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:50.090129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:50.090129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:50.091222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:50.091222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:50.091222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:44:50.091222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:44:50.091222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:44:50.091222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:44:50.091222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:44:50.091222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:44:50.091222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 10:44:50.091222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:50.091222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 261/513] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:50.091222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99068d-8caa-4444-a119-6945bba8b5d7 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:50.096898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce7888-1a57-458c-9b76-79da5caa9924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:50.096898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce7888-1a57-458c-9b76-79da5caa9924 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:50.096898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce7888-1a57-458c-9b76-79da5caa9924 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:50.096898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce7888-1a57-458c-9b76-79da5caa9924 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:50.097302 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce7888-1a57-458c-9b76-79da5caa9924 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:50.097302 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce7888-1a57-458c-9b76-79da5caa9924 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:50.097660 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce7888-1a57-458c-9b76-79da5caa9924 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:50.111825 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 253/514] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:44:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:50.112720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce7888-1a57-458c-9b76-79da5caa9924 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:50.159632 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:50.209933 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99068d-8caa-4444-a119-6945bba8b5d7 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:50.227881 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce7888-1a57-458c-9b76-79da5caa9924 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:50.255299 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93895a-0c0a-453c-979a-10530054e2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:50.255903 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93895a-0c0a-453c-979a-10530054e2a4 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:50.256179 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93895a-0c0a-453c-979a-10530054e2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:50.256447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93895a-0c0a-453c-979a-10530054e2a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:50.265940 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1daa95-5148-4888-aa04-16b2d53eb85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:50.266584 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1daa95-5148-4888-aa04-16b2d53eb85d None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:50.266863 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1daa95-5148-4888-aa04-16b2d53eb85d None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:50.267133 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1daa95-5148-4888-aa04-16b2d53eb85d None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:50.267779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1daa95-5148-4888-aa04-16b2d53eb85d None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:50.268079 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1daa95-5148-4888-aa04-16b2d53eb85d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:50.269521 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1daa95-5148-4888-aa04-16b2d53eb85d None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:50.281844 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 254/515] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:44:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:50.282836 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1daa95-5148-4888-aa04-16b2d53eb85d None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:50.296441 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:50.335363 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c93895a-0c0a-453c-979a-10530054e2a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:50.337011 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c93895a-0c0a-453c-979a-10530054e2a4 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:50.393547 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:50.398402 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1daa95-5148-4888-aa04-16b2d53eb85d None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:50.438934 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f68230-17fc-40aa-a303-a24aed532696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:50.439357 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f68230-17fc-40aa-a303-a24aed532696 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:50.439637 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f68230-17fc-40aa-a303-a24aed532696 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:50.439969 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f68230-17fc-40aa-a303-a24aed532696 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:50.440580 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f68230-17fc-40aa-a303-a24aed532696 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:50.440878 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f68230-17fc-40aa-a303-a24aed532696 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:50.442253 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f68230-17fc-40aa-a303-a24aed532696 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:50.452901 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 262/516] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:44:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:44:50.453753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93895a-0c0a-453c-979a-10530054e2a4 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:50.461563 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 255/517] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:44:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:50.461563 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f68230-17fc-40aa-a303-a24aed532696 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:50.527604 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:50.566949 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93895a-0c0a-453c-979a-10530054e2a4 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:50.569884 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f68230-17fc-40aa-a303-a24aed532696 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:50.610496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a242dd-4d6d-4eca-b8d9-371807351c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:50.611232 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a242dd-4d6d-4eca-b8d9-371807351c50 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:50.611360 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5eea04-c460-43bd-bc93-1c46e42f1827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:50.611508 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a242dd-4d6d-4eca-b8d9-371807351c50 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:50.611778 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a242dd-4d6d-4eca-b8d9-371807351c50 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:50.611971 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5eea04-c460-43bd-bc93-1c46e42f1827 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:50.612249 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5eea04-c460-43bd-bc93-1c46e42f1827 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:50.612523 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5eea04-c460-43bd-bc93-1c46e42f1827 None admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:50.613262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5eea04-c460-43bd-bc93-1c46e42f1827 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:50.613573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5eea04-c460-43bd-bc93-1c46e42f1827 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:50.615045 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5eea04-c460-43bd-bc93-1c46e42f1827 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:50.620352 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a242dd-4d6d-4eca-b8d9-371807351c50 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:50.621817 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a242dd-4d6d-4eca-b8d9-371807351c50 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:50.634165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-e1a242dd-4d6d-4eca-b8d9-371807351c50 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 10:44:50.634165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:50.634165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:50.634165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:50.634165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:50.634165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:50.634165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:50.634165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:50.634165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:50.634165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:50.634165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:50.634165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:50.634165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:44:50.634165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:44:50.634165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:44:50.634165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:44:50.635178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:50.635178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:50.635178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:44:50.635178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:44:50.635178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:44:50.635178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:44:50.635178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:50.635178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:50.635178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:50.635178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:50.635178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:50.635178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:50.635178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:50.635178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:50.635178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:50.635178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:50.635178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:50.635178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:50.636108 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:50.636108 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:50.636108 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:44:50.636108 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:44:50.636108 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:44:50.636108 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:44:50.636108 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:44:50.636108 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:44:50.636108 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 10:44:50.636108 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:50.636108 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 256/518] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 31 10:44:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:50.636108 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a242dd-4d6d-4eca-b8d9-371807351c50 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:50.653971 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 263/519] 10.222.0.16 () {56 vars in 1137 bytes} [Wed Aug 31 10:44:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:50.654860 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5eea04-c460-43bd-bc93-1c46e42f1827 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:50.667142 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:50.761216 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a242dd-4d6d-4eca-b8d9-371807351c50 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:50.774608 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5eea04-c460-43bd-bc93-1c46e42f1827 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:50.796557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f5c95d-c14e-4853-b67d-e50447f5237f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:50.797083 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5c95d-c14e-4853-b67d-e50447f5237f None admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:50.797360 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5c95d-c14e-4853-b67d-e50447f5237f None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:50.797633 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5c95d-c14e-4853-b67d-e50447f5237f None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:50.798278 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f5c95d-c14e-4853-b67d-e50447f5237f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:50.799759 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f5c95d-c14e-4853-b67d-e50447f5237f None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:50.799894 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:50.811124 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d89b5c-cb2a-4af6-a5bc-a32b0cd06e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:50.811759 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d89b5c-cb2a-4af6-a5bc-a32b0cd06e78 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:50.813188 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d89b5c-cb2a-4af6-a5bc-a32b0cd06e78 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:50.813188 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d89b5c-cb2a-4af6-a5bc-a32b0cd06e78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:50.822688 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d89b5c-cb2a-4af6-a5bc-a32b0cd06e78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:50.824327 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d89b5c-cb2a-4af6-a5bc-a32b0cd06e78 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:50.839637 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-e4d89b5c-cb2a-4af6-a5bc-a32b0cd06e78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:50.839637 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:50.839637 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:50.839637 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:50.839637 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:50.839637 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:50.839637 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:50.839637 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:50.839637 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:50.839637 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:50.839637 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:50.839637 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:50.839637 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:44:50.839637 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:44:50.839637 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:44:50.839637 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:44:50.842019 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:50.842019 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:50.842019 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:50.842019 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:50.842019 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:50.842019 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:50.842019 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:50.842019 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:50.842019 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:50.842019 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:50.842019 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:50.842019 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:50.842019 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:50.842019 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:50.842019 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:50.842019 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:50.842019 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:44:50.843083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:44:50.843083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:50.843083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:50.843083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 264/520] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:44:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:50.843083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d89b5c-cb2a-4af6-a5bc-a32b0cd06e78 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:50.902811 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 257/521] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 31 10:44:50 2022] POST /identity/v3/users => generated 247 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:44:50.904541 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f5c95d-c14e-4853-b67d-e50447f5237f None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:50.968913 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d89b5c-cb2a-4af6-a5bc-a32b0cd06e78 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:51.013319 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:51.020829 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae1a08-aa9b-4244-b401-877b4401ef59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:51.021579 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae1a08-aa9b-4244-b401-877b4401ef59 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:51.025492 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae1a08-aa9b-4244-b401-877b4401ef59 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:51.025492 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae1a08-aa9b-4244-b401-877b4401ef59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:51.031252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae1a08-aa9b-4244-b401-877b4401ef59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:51.033007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae1a08-aa9b-4244-b401-877b4401ef59 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:51.050217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-94ae1a08-aa9b-4244-b401-877b4401ef59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:44:51.050217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:51.050217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:51.050217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:51.050217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:51.050217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:51.050217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:51.050217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:51.050217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:51.050217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:51.050217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:51.050217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:51.050217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:44:51.050217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:44:51.050217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:44:51.051182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:44:51.051182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:51.051182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:51.051182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:51.051182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:51.051182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:51.051182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:51.051182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:51.051182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:51.051182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:51.051182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:51.051182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:51.051182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:51.051182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:51.051182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:51.051182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:51.051182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:51.052104 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:44:51.052104 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:44:51.052104 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:44:51.052104 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:44:51.052104 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:44:51.052104 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:44:51.052104 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:44:51.052104 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:51.052522 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f5c95d-c14e-4853-b67d-e50447f5237f None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:51.053752 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 265/522] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:44:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:51.054949 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae1a08-aa9b-4244-b401-877b4401ef59 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:51.089951 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb8ba2d-1d3d-410e-8589-9a69771411a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:51.091553 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8ba2d-1d3d-410e-8589-9a69771411a0 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:51.091981 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8ba2d-1d3d-410e-8589-9a69771411a0 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:51.092391 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8ba2d-1d3d-410e-8589-9a69771411a0 None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:51.093271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb8ba2d-1d3d-410e-8589-9a69771411a0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:51.093702 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb8ba2d-1d3d-410e-8589-9a69771411a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:51.095510 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb8ba2d-1d3d-410e-8589-9a69771411a0 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:51.109314 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:51.122727 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 258/523] 10.222.0.16 () {56 vars in 1139 bytes} [Wed Aug 31 10:44:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:51.123665 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb8ba2d-1d3d-410e-8589-9a69771411a0 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:51.190627 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae1a08-aa9b-4244-b401-877b4401ef59 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:51.235509 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b8147-d4de-4892-9f47-83cc4cc65921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:51.236735 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b8147-d4de-4892-9f47-83cc4cc65921 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:51.236735 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b8147-d4de-4892-9f47-83cc4cc65921 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:51.236998 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b8147-d4de-4892-9f47-83cc4cc65921 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:51.250653 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb8ba2d-1d3d-410e-8589-9a69771411a0 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:51.285989 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:51.291902 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83d2e6-2f37-4e7c-a375-5d9791ef8f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:51.292611 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83d2e6-2f37-4e7c-a375-5d9791ef8f9d None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:51.293048 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83d2e6-2f37-4e7c-a375-5d9791ef8f9d None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:51.293324 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83d2e6-2f37-4e7c-a375-5d9791ef8f9d None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:51.294449 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb83d2e6-2f37-4e7c-a375-5d9791ef8f9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:51.294726 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb83d2e6-2f37-4e7c-a375-5d9791ef8f9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:51.296116 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb83d2e6-2f37-4e7c-a375-5d9791ef8f9d None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:51.310431 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 259/524] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:44:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:51.310601 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83d2e6-2f37-4e7c-a375-5d9791ef8f9d None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:51.321972 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b8147-d4de-4892-9f47-83cc4cc65921 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:51.323493 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b8147-d4de-4892-9f47-83cc4cc65921 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:51.365862 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:51.435797 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83d2e6-2f37-4e7c-a375-5d9791ef8f9d None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:51.444990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 266/525] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:44:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:44:51.446403 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b8147-d4de-4892-9f47-83cc4cc65921 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:51.478155 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763e64a-ea5f-490e-a845-c6da71820cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:51.478790 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763e64a-ea5f-490e-a845-c6da71820cd9 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:51.479069 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763e64a-ea5f-490e-a845-c6da71820cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:51.479335 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763e64a-ea5f-490e-a845-c6da71820cd9 None admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:51.480031 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4763e64a-ea5f-490e-a845-c6da71820cd9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:51.480337 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4763e64a-ea5f-490e-a845-c6da71820cd9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:51.482265 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4763e64a-ea5f-490e-a845-c6da71820cd9 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:51.507142 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 260/526] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:44:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:51.507752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763e64a-ea5f-490e-a845-c6da71820cd9 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:51.576036 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:51.578945 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b8147-d4de-4892-9f47-83cc4cc65921 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:51.621481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0cb7ae-deb5-4390-8669-1693e854159f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:51.622798 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cb7ae-deb5-4390-8669-1693e854159f None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:51.623140 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cb7ae-deb5-4390-8669-1693e854159f None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:51.623414 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cb7ae-deb5-4390-8669-1693e854159f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:51.629315 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:51.634125 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763e64a-ea5f-490e-a845-c6da71820cd9 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:51.673072 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5016571-976a-46ef-98a0-22795b8f0522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:51.673710 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5016571-976a-46ef-98a0-22795b8f0522 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:51.673986 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5016571-976a-46ef-98a0-22795b8f0522 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:51.674254 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5016571-976a-46ef-98a0-22795b8f0522 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:51.682749 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5016571-976a-46ef-98a0-22795b8f0522 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:51.684145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5016571-976a-46ef-98a0-22795b8f0522 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:51.697823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-a5016571-976a-46ef-98a0-22795b8f0522 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:44:51.697823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:51.697823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:51.697823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:51.697823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:51.697823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:51.697823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:51.697823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:51.697823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:51.697823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:51.697823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:51.697823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:51.697823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:44:51.697823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:44:51.697823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:44:51.698903 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:44:51.698903 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:51.698903 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:51.698903 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:51.698903 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:51.698903 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:51.698903 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:51.698903 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:51.698903 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:51.698903 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:51.698903 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:51.698903 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:51.698903 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:51.698903 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:51.698903 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:51.698903 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:51.698903 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:51.699741 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:44:51.699741 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:44:51.699741 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:44:51.699741 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:44:51.699741 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:44:51.699741 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:44:51.699741 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:44:51.699741 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:51.699741 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 261/527] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:44:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:51.700267 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5016571-976a-46ef-98a0-22795b8f0522 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:51.711323 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0cb7ae-deb5-4390-8669-1693e854159f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:51.713076 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0cb7ae-deb5-4390-8669-1693e854159f None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:51.822779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5016571-976a-46ef-98a0-22795b8f0522 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:51.832121 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 267/528] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:44:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:44:51.833539 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0cb7ae-deb5-4390-8669-1693e854159f None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:51.842136 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:51.866322 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a0895-8ed3-4de1-af6d-eb2ed6bc6c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:51.866991 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a0895-8ed3-4de1-af6d-eb2ed6bc6c6a None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:51.867270 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a0895-8ed3-4de1-af6d-eb2ed6bc6c6a None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:51.867542 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a0895-8ed3-4de1-af6d-eb2ed6bc6c6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:51.876869 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:51.881048 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a0895-8ed3-4de1-af6d-eb2ed6bc6c6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:51.883490 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a0895-8ed3-4de1-af6d-eb2ed6bc6c6a None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:51.906914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-fa9a0895-8ed3-4de1-af6d-eb2ed6bc6c6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:51.906914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:51.906914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:51.906914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:51.906914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:51.906914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:51.906914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:51.906914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:51.906914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:51.906914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:51.906914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:51.906914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:51.906914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:44:51.906914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:44:51.906914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:44:51.906914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:44:51.912994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:51.912994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:51.912994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:51.912994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:51.912994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:51.912994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:51.912994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:51.912994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:51.912994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:51.912994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:51.912994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:51.912994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:51.912994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:51.912994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:51.912994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:51.912994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:51.912994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:44:51.914050 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:44:51.914050 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:51.914050 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:51.914050 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 262/529] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:44:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:51.914050 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a0895-8ed3-4de1-af6d-eb2ed6bc6c6a None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:51.970194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0cb7ae-deb5-4390-8669-1693e854159f None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:52.010798 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04df8d-70e9-4bf2-b08c-e0880319c484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:52.011502 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04df8d-70e9-4bf2-b08c-e0880319c484 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:52.011781 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04df8d-70e9-4bf2-b08c-e0880319c484 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:52.012052 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04df8d-70e9-4bf2-b08c-e0880319c484 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:52.012817 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04df8d-70e9-4bf2-b08c-e0880319c484 None admin] RBAC: Adding query filter params (role.id=bec755d6565a41798b09f5f57c1d51dd, scope.project.id=8db9b3f8a9f041938ac7b39003a35015, user.id=f11971a594014d86968c3356d11d42f7) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:52.013115 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04df8d-70e9-4bf2-b08c-e0880319c484 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:52.014601 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04df8d-70e9-4bf2-b08c-e0880319c484 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:52.020778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 268/530] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:44:51 2022] GET /identity/v3/role_assignments?user.id=f11971a594014d86968c3356d11d42f7&scope.project.id=8db9b3f8a9f041938ac7b39003a35015&role.id=bec755d6565a41798b09f5f57c1d51dd => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:52.021620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04df8d-70e9-4bf2-b08c-e0880319c484 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:52.056335 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a0895-8ed3-4de1-af6d-eb2ed6bc6c6a None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:52.112662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee947e1d-b064-45b5-bcd6-35036860c207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:52.113920 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee947e1d-b064-45b5-bcd6-35036860c207 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:52.114330 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee947e1d-b064-45b5-bcd6-35036860c207 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:52.114736 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee947e1d-b064-45b5-bcd6-35036860c207 None admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:52.116243 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee947e1d-b064-45b5-bcd6-35036860c207 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:52.116773 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee947e1d-b064-45b5-bcd6-35036860c207 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:52.118883 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee947e1d-b064-45b5-bcd6-35036860c207 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:52.138973 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 263/531] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:44:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:52.140506 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee947e1d-b064-45b5-bcd6-35036860c207 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:52.178272 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:52.186231 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:52.190303 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04df8d-70e9-4bf2-b08c-e0880319c484 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:52.243216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814e612-a650-431b-aa5f-48aa161b855e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:52.244077 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814e612-a650-431b-aa5f-48aa161b855e None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:52.244368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814e612-a650-431b-aa5f-48aa161b855e None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:52.244889 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814e612-a650-431b-aa5f-48aa161b855e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:52.258842 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814e612-a650-431b-aa5f-48aa161b855e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:52.260421 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814e612-a650-431b-aa5f-48aa161b855e None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:52.283796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-9814e612-a650-431b-aa5f-48aa161b855e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:52.283796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:52.283796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:52.283796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:52.283796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:52.283796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:52.283796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:52.283796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:52.283796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:52.283796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:52.283796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:52.283796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:52.283796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:44:52.283796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:44:52.283796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:44:52.283796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:44:52.284754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:52.284754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:52.284754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:52.284754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:52.284754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:52.284754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:52.284754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:52.284754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:52.284754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:52.284754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:52.284754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:52.284754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:52.284754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:52.284754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:52.284754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:52.284754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:52.284754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:44:52.285662 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:44:52.285662 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:52.285662 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:52.291227 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 269/532] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:44:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:52.292378 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814e612-a650-431b-aa5f-48aa161b855e None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:52.300684 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee947e1d-b064-45b5-bcd6-35036860c207 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:52.351325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca654ca1-af9e-4261-a402-865ef096dab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:52.353088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca654ca1-af9e-4261-a402-865ef096dab3 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:52.353588 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca654ca1-af9e-4261-a402-865ef096dab3 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:52.354055 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca654ca1-af9e-4261-a402-865ef096dab3 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:52.355554 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca654ca1-af9e-4261-a402-865ef096dab3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:52.356183 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca654ca1-af9e-4261-a402-865ef096dab3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:52.359413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca654ca1-af9e-4261-a402-865ef096dab3 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:52.377277 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 264/533] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:44:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:52.378655 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca654ca1-af9e-4261-a402-865ef096dab3 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:52.453182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814e612-a650-431b-aa5f-48aa161b855e None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:52.496740 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28701357-e2f1-474c-ac0a-4a7e47e2da1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:52.497990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28701357-e2f1-474c-ac0a-4a7e47e2da1d None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:52.498298 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28701357-e2f1-474c-ac0a-4a7e47e2da1d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:52.498671 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28701357-e2f1-474c-ac0a-4a7e47e2da1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:52.499470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28701357-e2f1-474c-ac0a-4a7e47e2da1d None admin] RBAC: Adding query filter params (role.id=bec755d6565a41798b09f5f57c1d51dd, scope.project.id=8db9b3f8a9f041938ac7b39003a35015, user.id=49f9aeb9857b45d68c3baa8fc4acc5fb) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:52.499784 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28701357-e2f1-474c-ac0a-4a7e47e2da1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:52.501370 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28701357-e2f1-474c-ac0a-4a7e47e2da1d None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:52.508924 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 270/534] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:44:52 2022] GET /identity/v3/role_assignments?user.id=49f9aeb9857b45d68c3baa8fc4acc5fb&scope.project.id=8db9b3f8a9f041938ac7b39003a35015&role.id=bec755d6565a41798b09f5f57c1d51dd => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:52.509567 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28701357-e2f1-474c-ac0a-4a7e47e2da1d None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:52.510100 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:52.534253 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca654ca1-af9e-4261-a402-865ef096dab3 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:52.536582 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:52.599273 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ade25d-026a-4b91-8e21-aa9636f3c9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:52.599913 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ade25d-026a-4b91-8e21-aa9636f3c9ba None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:52.600192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ade25d-026a-4b91-8e21-aa9636f3c9ba None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:52.600471 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ade25d-026a-4b91-8e21-aa9636f3c9ba None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:52.601146 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ade25d-026a-4b91-8e21-aa9636f3c9ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:52.601430 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ade25d-026a-4b91-8e21-aa9636f3c9ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:52.602986 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ade25d-026a-4b91-8e21-aa9636f3c9ba None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:52.643892 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 265/535] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:44:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:52.648618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ade25d-026a-4b91-8e21-aa9636f3c9ba None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:52.683233 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28701357-e2f1-474c-ac0a-4a7e47e2da1d None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:52.737079 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a2dc7-339c-4471-9faa-c2a8d435666d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:52.737978 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a2dc7-339c-4471-9faa-c2a8d435666d None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:52.738380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a2dc7-339c-4471-9faa-c2a8d435666d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:52.738778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a2dc7-339c-4471-9faa-c2a8d435666d None admin] PATH_INFO: `/v3/projects/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:52.754994 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771a2dc7-339c-4471-9faa-c2a8d435666d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:52.756808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771a2dc7-339c-4471-9faa-c2a8d435666d None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:52.788698 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-771a2dc7-339c-4471-9faa-c2a8d435666d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:44:52.788698 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:52.788698 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:52.788698 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:52.788698 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:52.788698 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:52.788698 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:52.788698 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:52.788698 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:52.788698 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:52.788698 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:52.788698 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:52.788698 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:44:52.788698 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:44:52.788698 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:44:52.789773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:44:52.789773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:52.789773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:52.789773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:52.789773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:52.789773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:52.789773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:52.789773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:52.789773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:52.789773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:52.789773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:52.789773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:52.789773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:52.789773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:52.789773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:52.789773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:52.789773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:52.790820 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:44:52.790820 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:44:52.790820 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:44:52.790820 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:44:52.790820 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:44:52.790820 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:44:52.790820 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:44:52.790820 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:52.791363 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 271/536] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:44:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:52.797664 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a2dc7-339c-4471-9faa-c2a8d435666d None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:52.799551 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ade25d-026a-4b91-8e21-aa9636f3c9ba None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:52.854946 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f81ff9-e068-40e7-be6b-acdd231eb082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:52.855871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f81ff9-e068-40e7-be6b-acdd231eb082 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:52.856773 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f81ff9-e068-40e7-be6b-acdd231eb082 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:52.857164 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f81ff9-e068-40e7-be6b-acdd231eb082 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:52.878466 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f81ff9-e068-40e7-be6b-acdd231eb082 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:52.879950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f81ff9-e068-40e7-be6b-acdd231eb082 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:52.902616 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-87f81ff9-e068-40e7-be6b-acdd231eb082 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:44:52.902616 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:52.902616 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:52.902616 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:52.902616 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:52.902616 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:52.902616 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:52.902616 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:52.902616 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:52.902616 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:52.902616 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:52.902616 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:52.902616 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:44:52.902616 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:44:52.902616 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:44:52.904133 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:44:52.904133 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:52.904133 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:52.904133 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:52.904133 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:52.904133 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:52.904133 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:52.904133 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:52.904133 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:52.904133 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:52.904133 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:52.904133 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:52.904133 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:52.904133 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:52.904133 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:52.904133 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:52.904133 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:52.905437 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:44:52.905437 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:44:52.905437 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:44:52.905437 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:44:52.905437 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:44:52.905437 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:44:52.905437 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:44:52.905437 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:52.905437 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 266/537] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:44:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:52.915434 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f81ff9-e068-40e7-be6b-acdd231eb082 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:52.928616 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:52.930019 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:52.968239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a2dc7-339c-4471-9faa-c2a8d435666d None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:53.042546 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917c14d-1cc4-4fb4-81ed-0e4378a3f8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:53.043312 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917c14d-1cc4-4fb4-81ed-0e4378a3f8b0 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:53.043685 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917c14d-1cc4-4fb4-81ed-0e4378a3f8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:53.044002 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917c14d-1cc4-4fb4-81ed-0e4378a3f8b0 None admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:53.044945 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917c14d-1cc4-4fb4-81ed-0e4378a3f8b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:53.045258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917c14d-1cc4-4fb4-81ed-0e4378a3f8b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:53.046750 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917c14d-1cc4-4fb4-81ed-0e4378a3f8b0 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:53.058338 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f81ff9-e068-40e7-be6b-acdd231eb082 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:53.067424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 272/538] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:44:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:44:53.068710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48400fa9-b290-4838-9f6f-ed45937c60eb None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:53.068998 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48400fa9-b290-4838-9f6f-ed45937c60eb None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:53.069277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48400fa9-b290-4838-9f6f-ed45937c60eb None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:53.076419 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 273/539] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:53.077335 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48400fa9-b290-4838-9f6f-ed45937c60eb None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:53.102459 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f540a1-e3b4-42ec-b616-58fa2d9564e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:53.102459 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f540a1-e3b4-42ec-b616-58fa2d9564e0 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:53.102459 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f540a1-e3b4-42ec-b616-58fa2d9564e0 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:53.102459 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f540a1-e3b4-42ec-b616-58fa2d9564e0 None admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:53.102975 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f540a1-e3b4-42ec-b616-58fa2d9564e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:53.102975 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f540a1-e3b4-42ec-b616-58fa2d9564e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:53.102975 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f540a1-e3b4-42ec-b616-58fa2d9564e0 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:53.124244 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 267/540] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:44:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:53.124244 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7dc5d-afdd-488f-9517-0cc8d90b309b None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:53.124244 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7dc5d-afdd-488f-9517-0cc8d90b309b None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:53.124244 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7dc5d-afdd-488f-9517-0cc8d90b309b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:53.143345 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-4be7dc5d-afdd-488f-9517-0cc8d90b309b None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:53.144754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-4be7dc5d-afdd-488f-9517-0cc8d90b309b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:53.198564 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48400fa9-b290-4838-9f6f-ed45937c60eb None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:53.214789 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:53.222815 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 268/541] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:53.223754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be7dc5d-afdd-488f-9517-0cc8d90b309b None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:53.238282 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc053a1-3a2c-46cf-bf43-416323a4be39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:53.239230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc053a1-3a2c-46cf-bf43-416323a4be39 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:53.239656 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc053a1-3a2c-46cf-bf43-416323a4be39 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:53.240117 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc053a1-3a2c-46cf-bf43-416323a4be39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:53.241013 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc053a1-3a2c-46cf-bf43-416323a4be39 None admin] RBAC: Adding query filter params (role.id=bec755d6565a41798b09f5f57c1d51dd, scope.project.id=8db9b3f8a9f041938ac7b39003a35015, user.id=f2e72c8e7b444e978bf99c6404febcd2) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:53.241486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc053a1-3a2c-46cf-bf43-416323a4be39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:53.243222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc053a1-3a2c-46cf-bf43-416323a4be39 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:53.247887 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:53.251459 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 274/542] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:44:53 2022] GET /identity/v3/role_assignments?user.id=f2e72c8e7b444e978bf99c6404febcd2&scope.project.id=8db9b3f8a9f041938ac7b39003a35015&role.id=bec755d6565a41798b09f5f57c1d51dd => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:53.413588 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be7dc5d-afdd-488f-9517-0cc8d90b309b None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:53.452451 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74993e3-ed8d-46e4-8180-fa49bbd2499a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:53.452997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74993e3-ed8d-46e4-8180-fa49bbd2499a None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:53.453275 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74993e3-ed8d-46e4-8180-fa49bbd2499a None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:53.453547 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74993e3-ed8d-46e4-8180-fa49bbd2499a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:53.454451 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74993e3-ed8d-46e4-8180-fa49bbd2499a None admin] RBAC: Adding query filter params (role.id=bec755d6565a41798b09f5f57c1d51dd, scope.project.id=8db9b3f8a9f041938ac7b39003a35015, user.id=1a00b7f45b2e47b8a01048c36d00f742) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:53.454766 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74993e3-ed8d-46e4-8180-fa49bbd2499a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:53.456215 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74993e3-ed8d-46e4-8180-fa49bbd2499a None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:53.463689 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 269/543] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:44:53 2022] GET /identity/v3/role_assignments?user.id=1a00b7f45b2e47b8a01048c36d00f742&scope.project.id=8db9b3f8a9f041938ac7b39003a35015&role.id=bec755d6565a41798b09f5f57c1d51dd => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:53.610492 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc053a1-3a2c-46cf-bf43-416323a4be39 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:53.729854 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc053a1-3a2c-46cf-bf43-416323a4be39 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:53.783189 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f8830-f138-47e5-9741-4941aeadfee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:53.783891 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f8830-f138-47e5-9741-4941aeadfee2 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:53.784167 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f8830-f138-47e5-9741-4941aeadfee2 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:53.784435 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f8830-f138-47e5-9741-4941aeadfee2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:53.796266 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763f8830-f138-47e5-9741-4941aeadfee2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:53.805923 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-763f8830-f138-47e5-9741-4941aeadfee2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:44:53.805923 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:53.805923 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:53.805923 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:53.805923 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:53.805923 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:53.805923 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:53.805923 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:53.805923 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:53.805923 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:53.805923 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:53.805923 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:53.805923 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:53.805923 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:53.805923 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:53.805923 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:53.806886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:53.806886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:53.806886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:53.806886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:53.806886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:53.806886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:53.806886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:53.806886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:53.806886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:53.806886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:53.806886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:53.806886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:53.806886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:53.806886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:53.806886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:53.806886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:53.807648 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:53.807648 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:53.807648 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:53.807648 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:53.807648 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:53.807648 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:53.807648 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:53.807648 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:53.807648 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:53.807648 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:53.807648 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:44:53.807648 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:53.807648 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 275/544] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 31 10:44:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:53.813653 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74993e3-ed8d-46e4-8180-fa49bbd2499a None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:53.928269 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:53.992623 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74993e3-ed8d-46e4-8180-fa49bbd2499a None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:54.043396 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3d70d-f890-4ac8-8a49-6a714184cb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:54.045217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3d70d-f890-4ac8-8a49-6a714184cb56 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:54.045503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3d70d-f890-4ac8-8a49-6a714184cb56 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:54.045775 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3d70d-f890-4ac8-8a49-6a714184cb56 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:54.046482 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce3d70d-f890-4ac8-8a49-6a714184cb56 None admin] RBAC: Adding query filter params (name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:54.046786 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce3d70d-f890-4ac8-8a49-6a714184cb56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:54.048176 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce3d70d-f890-4ac8-8a49-6a714184cb56 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:54.063103 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 270/545] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 31 10:44:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:54.068249 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f8830-f138-47e5-9741-4941aeadfee2 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:54.163321 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:54.217672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f8830-f138-47e5-9741-4941aeadfee2 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:54.258180 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4eddf-9d67-41a0-be8b-098c023a3aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:54.259216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4eddf-9d67-41a0-be8b-098c023a3aaa None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:54.259718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4eddf-9d67-41a0-be8b-098c023a3aaa None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:54.260194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4eddf-9d67-41a0-be8b-098c023a3aaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:54.275272 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e4eddf-9d67-41a0-be8b-098c023a3aaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:54.277063 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e4eddf-9d67-41a0-be8b-098c023a3aaa None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:54.293778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-a1e4eddf-9d67-41a0-be8b-098c023a3aaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:54.293778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:54.293778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:54.293778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:54.293778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:54.293778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:54.293778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:54.293778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:54.293778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:54.293778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:54.293778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:54.293778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:54.293778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:44:54.293778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:44:54.293778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:44:54.293778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:44:54.295596 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:54.295596 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:54.295596 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:54.295596 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:54.295596 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:54.295596 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:54.295596 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:54.295596 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:54.295596 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:54.295596 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:54.295596 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:54.295596 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:54.295596 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:54.295596 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:54.295596 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:54.295596 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:54.295596 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:44:54.296593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:44:54.296593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:54.296593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:54.299479 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 276/546] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:44:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:54.307525 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3d70d-f890-4ac8-8a49-6a714184cb56 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:54.459544 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3d70d-f890-4ac8-8a49-6a714184cb56 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:54.483511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:54.507130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e7e34-2569-479b-969d-d278dc2ea90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:54.507764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e7e34-2569-479b-969d-d278dc2ea90c None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:54.508071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e7e34-2569-479b-969d-d278dc2ea90c None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:54.508400 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e7e34-2569-479b-969d-d278dc2ea90c None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:54.509252 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e7e34-2569-479b-969d-d278dc2ea90c None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:54.509793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e7e34-2569-479b-969d-d278dc2ea90c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:54.512182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e7e34-2569-479b-969d-d278dc2ea90c None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:54.535136 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 271/547] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:44:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:54.540272 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4eddf-9d67-41a0-be8b-098c023a3aaa None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:54.618874 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f1aa0-57be-4b31-9b70-3b5432cedee2 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:54.619250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f1aa0-57be-4b31-9b70-3b5432cedee2 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:54.619418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f1aa0-57be-4b31-9b70-3b5432cedee2 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:54.621365 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 272/548] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:44:54.627681 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf87ff3-1399-48bc-a8ba-9c901e82df48 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:54.627987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf87ff3-1399-48bc-a8ba-9c901e82df48 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:54.628268 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf87ff3-1399-48bc-a8ba-9c901e82df48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:54.675076 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-bcf87ff3-1399-48bc-a8ba-9c901e82df48 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:54.678593 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-bcf87ff3-1399-48bc-a8ba-9c901e82df48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:54.722684 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4eddf-9d67-41a0-be8b-098c023a3aaa None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:54.738030 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:54.764115 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a572c23-4a61-49c8-838e-0b7c99acf285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:54.765197 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a572c23-4a61-49c8-838e-0b7c99acf285 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:54.765676 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a572c23-4a61-49c8-838e-0b7c99acf285 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:54.766425 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a572c23-4a61-49c8-838e-0b7c99acf285 None admin] PATH_INFO: `/v3/users/swift` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:54.776381 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a572c23-4a61-49c8-838e-0b7c99acf285 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:54.778314 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a572c23-4a61-49c8-838e-0b7c99acf285 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:54.791513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-9a572c23-4a61-49c8-838e-0b7c99acf285 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 10:44:54.791513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:54.791513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:54.791513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:54.791513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:54.791513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:54.791513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:54.791513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:54.791513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:54.791513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:54.791513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:54.791513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:54.791513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:44:54.791513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:44:54.791513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:44:54.791513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:44:54.792515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:54.792515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:54.792515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:44:54.792515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:44:54.792515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:44:54.792515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:44:54.792515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:54.792515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:54.792515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:54.792515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:54.792515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:54.792515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:54.792515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:54.792515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:54.792515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:54.792515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:54.792515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:54.792515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:54.793415 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:54.793415 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:54.793415 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:44:54.793415 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:44:54.793415 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:44:54.793415 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:44:54.793415 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:44:54.793415 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:44:54.793415 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 10:44:54.793415 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:54.796743 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 277/549] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:44:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:54.802737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a572c23-4a61-49c8-838e-0b7c99acf285 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:54.882961 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 273/550] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:44:54.952264 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa730d8d-98be-45c5-99cb-7e9a5b082a51 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:54.953005 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa730d8d-98be-45c5-99cb-7e9a5b082a51 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:54.953104 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa730d8d-98be-45c5-99cb-7e9a5b082a51 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:54.957147 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a572c23-4a61-49c8-838e-0b7c99acf285 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:54.963370 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 274/551] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:54 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:44:54.969033 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab5841-0b06-4dbe-b6ee-3fbfb396e03b None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:54.969313 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab5841-0b06-4dbe-b6ee-3fbfb396e03b None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:54.969589 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab5841-0b06-4dbe-b6ee-3fbfb396e03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:54.998695 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6b29a-c74f-4809-9b3f-c1397d505fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:54.999710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6b29a-c74f-4809-9b3f-c1397d505fed None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:55.000323 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6b29a-c74f-4809-9b3f-c1397d505fed None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:55.000796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6b29a-c74f-4809-9b3f-c1397d505fed None admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:55.001818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb6b29a-c74f-4809-9b3f-c1397d505fed None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:55.002368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb6b29a-c74f-4809-9b3f-c1397d505fed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:55.004100 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb6b29a-c74f-4809-9b3f-c1397d505fed None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:55.016221 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-41ab5841-0b06-4dbe-b6ee-3fbfb396e03b None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:55.016754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-41ab5841-0b06-4dbe-b6ee-3fbfb396e03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:55.025710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 278/552] 10.222.0.16 () {56 vars in 1135 bytes} [Wed Aug 31 10:44:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:55.034234 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6b29a-c74f-4809-9b3f-c1397d505fed None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:55.069530 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:55.155467 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6b29a-c74f-4809-9b3f-c1397d505fed None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:55.194049 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a685d57-b359-453b-9ff5-e6c4e4052ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:55.195090 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a685d57-b359-453b-9ff5-e6c4e4052ebb None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:55.195563 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a685d57-b359-453b-9ff5-e6c4e4052ebb None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:55.196459 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a685d57-b359-453b-9ff5-e6c4e4052ebb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:55.214582 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 275/553] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:55.216720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab5841-0b06-4dbe-b6ee-3fbfb396e03b None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:55.273103 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a685d57-b359-453b-9ff5-e6c4e4052ebb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:55.274504 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a685d57-b359-453b-9ff5-e6c4e4052ebb None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:55.298484 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:55.308151 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:55.415496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab5841-0b06-4dbe-b6ee-3fbfb396e03b None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:55.463352 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c4107-79e1-4248-9c2b-7e8c9b07643e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:55.464383 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c4107-79e1-4248-9c2b-7e8c9b07643e None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:55.464862 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c4107-79e1-4248-9c2b-7e8c9b07643e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:55.465327 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c4107-79e1-4248-9c2b-7e8c9b07643e None admin] PATH_INFO: `/v3/roles/service` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:55.474535 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 279/554] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:44:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:44:55.479998 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a685d57-b359-453b-9ff5-e6c4e4052ebb None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:55.495143 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4c4107-79e1-4248-9c2b-7e8c9b07643e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:55.500856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-8e4c4107-79e1-4248-9c2b-7e8c9b07643e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:44:55.500856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:55.500856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:55.500856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:55.500856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:55.500856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:55.500856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:55.500856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:55.500856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:55.500856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:55.500856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:55.500856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:55.500856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:55.500856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:55.500856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:55.500856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:55.501816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:55.501816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:55.501816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:55.501816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:55.501816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:55.501816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:55.501816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:55.501816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:55.501816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:55.501816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:55.501816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:55.501816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:55.501816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:55.501816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:55.501816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:55.501816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:55.502658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:55.502658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:55.502658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:55.502658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:55.502658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:55.502658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:55.502658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:55.502658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:55.502658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:55.502658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:55.502658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:44:55.502658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:55.513176 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 276/555] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 31 10:44:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:55.519423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c4107-79e1-4248-9c2b-7e8c9b07643e None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:55.606787 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a685d57-b359-453b-9ff5-e6c4e4052ebb None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:55.641552 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:55.654172 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef2bf6-96ed-488c-82bd-5867ccb7cb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:55.654172 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef2bf6-96ed-488c-82bd-5867ccb7cb7e None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:55.654172 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef2bf6-96ed-488c-82bd-5867ccb7cb7e None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:55.654172 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef2bf6-96ed-488c-82bd-5867ccb7cb7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:55.657509 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:55.658512 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c4107-79e1-4248-9c2b-7e8c9b07643e None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:55.666807 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef2bf6-96ed-488c-82bd-5867ccb7cb7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:55.668288 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef2bf6-96ed-488c-82bd-5867ccb7cb7e None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:55.685173 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-55ef2bf6-96ed-488c-82bd-5867ccb7cb7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:55.685173 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:55.685173 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:55.685173 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:55.685173 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:55.685173 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:55.685173 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:55.685173 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:55.685173 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:55.685173 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:55.685173 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:55.685173 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:55.685173 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:44:55.685173 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:44:55.685173 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:44:55.685173 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:44:55.686350 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:55.686350 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:55.686350 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:55.686350 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:55.686350 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:55.686350 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:55.686350 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:55.686350 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:55.686350 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:55.686350 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:55.686350 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:55.686350 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:55.686350 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:55.686350 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:55.686350 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:55.686350 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:55.686350 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:44:55.687942 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:44:55.687942 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:55.687942 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:55.687942 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 280/556] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:44:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:55.687942 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef2bf6-96ed-488c-82bd-5867ccb7cb7e None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:55.705191 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624af429-e48d-48fe-8b2a-a3f65d2837a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:55.706817 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624af429-e48d-48fe-8b2a-a3f65d2837a0 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:55.707268 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624af429-e48d-48fe-8b2a-a3f65d2837a0 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:55.707719 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624af429-e48d-48fe-8b2a-a3f65d2837a0 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:55.708777 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624af429-e48d-48fe-8b2a-a3f65d2837a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:55.709249 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624af429-e48d-48fe-8b2a-a3f65d2837a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:55.711430 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624af429-e48d-48fe-8b2a-a3f65d2837a0 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:55.725789 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 277/557] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 31 10:44:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:55.725789 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9906a-837e-4b84-ba27-a9df5227bed9 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:55.725789 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9906a-837e-4b84-ba27-a9df5227bed9 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:55.725789 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9906a-837e-4b84-ba27-a9df5227bed9 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:55.729164 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 278/558] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:55.730048 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9906a-837e-4b84-ba27-a9df5227bed9 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:55.807446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef2bf6-96ed-488c-82bd-5867ccb7cb7e None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:55.857489 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2d10b-5d27-4a0a-b7a8-a36c58296a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:55.857894 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2d10b-5d27-4a0a-b7a8-a36c58296a06 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:55.857998 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9906a-837e-4b84-ba27-a9df5227bed9 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:55.858161 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2d10b-5d27-4a0a-b7a8-a36c58296a06 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:55.858458 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2d10b-5d27-4a0a-b7a8-a36c58296a06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:55.870010 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2d10b-5d27-4a0a-b7a8-a36c58296a06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:55.875888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-b8a2d10b-5d27-4a0a-b7a8-a36c58296a06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:44:55.875888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:55.875888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:55.875888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:55.875888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:55.875888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:55.875888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:55.875888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:55.875888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:55.875888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:55.875888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:55.875888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:55.875888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:55.875888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:55.875888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:55.875888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:55.876948 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:55.876948 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:55.876948 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:55.876948 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:55.876948 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:55.876948 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:55.876948 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:55.876948 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:55.876948 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:55.876948 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:55.876948 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:55.876948 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:55.876948 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:55.876948 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:55.876948 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:55.876948 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:55.877776 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:55.877776 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:55.877776 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:55.877776 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:55.877776 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:55.877776 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:55.877776 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:55.877776 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:55.877776 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:55.877776 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:55.877776 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:44:55.877776 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:55.877776 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 281/559] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 31 10:44:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:55.877776 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2d10b-5d27-4a0a-b7a8-a36c58296a06 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:55.904490 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3180567-318f-4a26-b241-7edcd2c0f38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:55.905122 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3180567-318f-4a26-b241-7edcd2c0f38a None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:55.905401 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3180567-318f-4a26-b241-7edcd2c0f38a None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:55.905678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3180567-318f-4a26-b241-7edcd2c0f38a None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:55.906330 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3180567-318f-4a26-b241-7edcd2c0f38a None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:55.906697 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3180567-318f-4a26-b241-7edcd2c0f38a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:55.908088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3180567-318f-4a26-b241-7edcd2c0f38a None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:55.923795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 279/560] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:44:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:55.925300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacea8f1-55ff-4556-9ff9-3afc4e8d62cc None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:55.926045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacea8f1-55ff-4556-9ff9-3afc4e8d62cc None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:55.926488 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacea8f1-55ff-4556-9ff9-3afc4e8d62cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:55.931794 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:55.948308 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-dacea8f1-55ff-4556-9ff9-3afc4e8d62cc None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:55.951297 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-dacea8f1-55ff-4556-9ff9-3afc4e8d62cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:55.968826 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:56.004769 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2d10b-5d27-4a0a-b7a8-a36c58296a06 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:56.024877 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 280/561] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:56.025726 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacea8f1-55ff-4556-9ff9-3afc4e8d62cc None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:56.041098 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001a25a-1e3f-4f86-9002-dcca61d9f42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:56.041729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001a25a-1e3f-4f86-9002-dcca61d9f42e None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:56.042074 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001a25a-1e3f-4f86-9002-dcca61d9f42e None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:56.042343 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001a25a-1e3f-4f86-9002-dcca61d9f42e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:56.054060 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001a25a-1e3f-4f86-9002-dcca61d9f42e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:56.056295 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001a25a-1e3f-4f86-9002-dcca61d9f42e None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:56.071537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-4001a25a-1e3f-4f86-9002-dcca61d9f42e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:56.071537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:56.071537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:56.071537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:56.071537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:56.071537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:56.071537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:56.071537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:56.071537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:56.071537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:56.071537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:56.071537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:56.071537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:44:56.071537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:44:56.071537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:44:56.071537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:44:56.072980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:56.072980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:56.072980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:56.072980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:56.072980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:56.072980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:56.072980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:56.072980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:56.072980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:56.072980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:56.072980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:56.072980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:56.072980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:56.072980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:56.072980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:56.072980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:56.072980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:44:56.073840 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:44:56.073840 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:56.073840 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:56.073840 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 282/562] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:44:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:56.073840 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001a25a-1e3f-4f86-9002-dcca61d9f42e None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:56.192537 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacea8f1-55ff-4556-9ff9-3afc4e8d62cc None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:56.202749 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:56.209469 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001a25a-1e3f-4f86-9002-dcca61d9f42e None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:56.239713 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a4ef5-f81f-471a-be85-eb82b1e58621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:56.240372 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a4ef5-f81f-471a-be85-eb82b1e58621 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:56.240709 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a4ef5-f81f-471a-be85-eb82b1e58621 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:56.240954 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a4ef5-f81f-471a-be85-eb82b1e58621 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:56.241626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a4ef5-f81f-471a-be85-eb82b1e58621 None admin] RBAC: Adding query filter params (name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:56.242035 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a4ef5-f81f-471a-be85-eb82b1e58621 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:56.243629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a4ef5-f81f-471a-be85-eb82b1e58621 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:56.245144 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37d0dc-bd2b-4c95-9c00-823e3aa56638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:56.245799 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37d0dc-bd2b-4c95-9c00-823e3aa56638 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:56.246078 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37d0dc-bd2b-4c95-9c00-823e3aa56638 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:56.246345 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37d0dc-bd2b-4c95-9c00-823e3aa56638 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:56.257519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f37d0dc-bd2b-4c95-9c00-823e3aa56638 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:56.264094 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 281/563] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 31 10:44:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:56.264265 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f37d0dc-bd2b-4c95-9c00-823e3aa56638 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:56.264368 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b80c1-57e0-4dc1-9539-c33cb5b6397e None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:56.264368 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b80c1-57e0-4dc1-9539-c33cb5b6397e None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:56.264368 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b80c1-57e0-4dc1-9539-c33cb5b6397e None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:56.264368 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 282/564] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:56.264368 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b80c1-57e0-4dc1-9539-c33cb5b6397e None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:56.278939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-1f37d0dc-bd2b-4c95-9c00-823e3aa56638 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:44:56.278939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:56.278939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:56.278939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:56.278939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:56.278939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:56.278939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:56.278939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:56.278939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:56.278939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:56.278939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:56.278939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:56.278939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:44:56.278939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:44:56.278939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:44:56.279881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:44:56.279881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:56.279881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:56.279881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:56.279881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:56.279881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:56.279881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:56.279881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:56.279881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:56.279881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:56.279881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:56.279881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:56.279881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:56.279881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:56.279881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:56.279881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:56.279881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:56.281238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:44:56.281238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:44:56.281238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:44:56.281238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:44:56.281238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:44:56.281238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:44:56.281238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:44:56.281238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:56.281238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 283/565] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:44:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:56.281238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008036d-fb5f-4c47-9cd5-25d8b84ab379 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:56.281238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008036d-fb5f-4c47-9cd5-25d8b84ab379 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:56.281831 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008036d-fb5f-4c47-9cd5-25d8b84ab379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:56.286523 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:56.300676 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-e008036d-fb5f-4c47-9cd5-25d8b84ab379 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:56.301913 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-e008036d-fb5f-4c47-9cd5-25d8b84ab379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:56.393753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 284/566] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:56.394537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008036d-fb5f-4c47-9cd5-25d8b84ab379 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:56.399797 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b80c1-57e0-4dc1-9539-c33cb5b6397e None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:56.439453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026faf7-008c-44d3-816b-6431ade994b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:56.440063 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026faf7-008c-44d3-816b-6431ade994b2 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:56.440341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026faf7-008c-44d3-816b-6431ade994b2 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:56.440609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026faf7-008c-44d3-816b-6431ade994b2 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:56.441254 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c026faf7-008c-44d3-816b-6431ade994b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:56.441931 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c026faf7-008c-44d3-816b-6431ade994b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:56.442952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c026faf7-008c-44d3-816b-6431ade994b2 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:56.456799 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 283/567] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:56.457474 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026faf7-008c-44d3-816b-6431ade994b2 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:56.504120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:56.521956 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008036d-fb5f-4c47-9cd5-25d8b84ab379 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:56.546602 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:56.561986 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd8c0b-7e61-4d97-b5ce-22f8679571e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:56.562647 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd8c0b-7e61-4d97-b5ce-22f8679571e4 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:56.563452 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd8c0b-7e61-4d97-b5ce-22f8679571e4 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:56.563729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd8c0b-7e61-4d97-b5ce-22f8679571e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:56.575533 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026faf7-008c-44d3-816b-6431ade994b2 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:56.576400 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd8c0b-7e61-4d97-b5ce-22f8679571e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:56.577890 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd8c0b-7e61-4d97-b5ce-22f8679571e4 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:56.593868 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-04dd8c0b-7e61-4d97-b5ce-22f8679571e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:56.593868 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:56.593868 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:56.593868 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:56.593868 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:56.593868 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:56.593868 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:56.593868 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:56.593868 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:56.593868 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:56.593868 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:56.593868 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:56.593868 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:44:56.593868 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:44:56.593868 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:44:56.593868 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:44:56.594921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:56.594921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:56.594921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:56.594921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:56.594921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:56.594921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:56.594921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:56.594921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:56.594921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:56.594921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:56.594921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:56.594921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:56.594921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:56.594921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:56.594921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:56.594921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:56.594921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:44:56.595762 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:44:56.595762 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:56.595762 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:56.599822 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 285/568] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:44:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:56.600587 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd8c0b-7e61-4d97-b5ce-22f8679571e4 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:56.625777 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e494573-78f0-4444-a81b-dea61b42c309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:56.626565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e494573-78f0-4444-a81b-dea61b42c309 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:56.626836 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e494573-78f0-4444-a81b-dea61b42c309 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:56.627099 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e494573-78f0-4444-a81b-dea61b42c309 None admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:56.627789 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e494573-78f0-4444-a81b-dea61b42c309 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:56.628113 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e494573-78f0-4444-a81b-dea61b42c309 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:56.629832 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e494573-78f0-4444-a81b-dea61b42c309 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:56.643448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 284/569] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:44:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:56.644710 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e494573-78f0-4444-a81b-dea61b42c309 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:56.723985 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd8c0b-7e61-4d97-b5ce-22f8679571e4 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:56.757704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e494573-78f0-4444-a81b-dea61b42c309 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:56.767127 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea5a1e-e001-45fd-83a6-a00003acd845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:56.771432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea5a1e-e001-45fd-83a6-a00003acd845 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:56.771432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea5a1e-e001-45fd-83a6-a00003acd845 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:56.771432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea5a1e-e001-45fd-83a6-a00003acd845 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:56.774173 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:56.783774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea5a1e-e001-45fd-83a6-a00003acd845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:56.788937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-32ea5a1e-e001-45fd-83a6-a00003acd845 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:44:56.788937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:56.788937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:56.788937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:56.788937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:56.788937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:56.788937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:56.788937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:56.788937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:56.788937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:56.788937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:56.788937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:56.788937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:56.788937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:56.788937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:56.788937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:56.789915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:56.789915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:56.789915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:56.789915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:56.789915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:56.789915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:56.789915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:56.789915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:56.789915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:56.789915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:56.789915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:56.789915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:56.789915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:56.789915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:56.789915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:56.789915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:56.790975 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:56.790975 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:56.790975 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:56.790975 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:56.790975 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:56.790975 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:56.790975 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:56.790975 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:56.790975 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:56.790975 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:56.790975 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:44:56.790975 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:56.790975 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 286/570] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 31 10:44:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:56.790975 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea5a1e-e001-45fd-83a6-a00003acd845 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:56.801704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89486da6-5990-4e23-8d2f-21d733900eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:56.802339 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89486da6-5990-4e23-8d2f-21d733900eef None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:56.802673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89486da6-5990-4e23-8d2f-21d733900eef None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:56.802902 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89486da6-5990-4e23-8d2f-21d733900eef None admin] PATH_INFO: `/v3/users/nova` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:56.805933 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:56.814106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89486da6-5990-4e23-8d2f-21d733900eef None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:56.815697 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89486da6-5990-4e23-8d2f-21d733900eef None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:56.828611 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-89486da6-5990-4e23-8d2f-21d733900eef None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:44:56.828611 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:56.828611 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:56.828611 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:56.828611 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:56.828611 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:56.828611 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:56.828611 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:56.828611 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:56.828611 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:56.828611 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:56.828611 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:56.828611 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:44:56.828611 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:44:56.828611 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:44:56.828611 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:44:56.829746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:56.829746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:56.829746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:44:56.829746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:44:56.829746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:44:56.829746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:44:56.829746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:56.829746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:56.829746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:56.829746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:56.829746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:56.829746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:56.829746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:56.829746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:56.829746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:56.829746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:56.829746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:56.829746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:56.830700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:56.830700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:56.830700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:44:56.830700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:44:56.830700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:44:56.830700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:44:56.830700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:44:56.830700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:44:56.830700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:44:56.830700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:56.830700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 285/571] 10.222.0.16 () {56 vars in 1093 bytes} [Wed Aug 31 10:44:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:56.830700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89486da6-5990-4e23-8d2f-21d733900eef None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:56.903932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea5a1e-e001-45fd-83a6-a00003acd845 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:56.940695 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bfa5f3-1889-49b3-8a0e-96504c5a4ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:56.941295 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bfa5f3-1889-49b3-8a0e-96504c5a4ae3 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:56.941569 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bfa5f3-1889-49b3-8a0e-96504c5a4ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:56.941839 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bfa5f3-1889-49b3-8a0e-96504c5a4ae3 None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:56.942499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bfa5f3-1889-49b3-8a0e-96504c5a4ae3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:56.942796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bfa5f3-1889-49b3-8a0e-96504c5a4ae3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:56.944167 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bfa5f3-1889-49b3-8a0e-96504c5a4ae3 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:56.944699 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89486da6-5990-4e23-8d2f-21d733900eef None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:56.957194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 287/572] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:56.958031 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bfa5f3-1889-49b3-8a0e-96504c5a4ae3 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:56.983490 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4767d17a-8766-4672-b2c6-028c1a078970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:56.984226 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767d17a-8766-4672-b2c6-028c1a078970 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:56.984507 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767d17a-8766-4672-b2c6-028c1a078970 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:56.984779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767d17a-8766-4672-b2c6-028c1a078970 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:56.985533 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4767d17a-8766-4672-b2c6-028c1a078970 None admin] RBAC: Adding query filter params (role.id=bec755d6565a41798b09f5f57c1d51dd, scope.project.id=8db9b3f8a9f041938ac7b39003a35015, user.id=e8abc7c439a04622896b34ed0a7b8492) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:56.985870 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4767d17a-8766-4672-b2c6-028c1a078970 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:56.987349 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4767d17a-8766-4672-b2c6-028c1a078970 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:56.999208 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 286/573] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:44:56 2022] GET /identity/v3/role_assignments?user.id=e8abc7c439a04622896b34ed0a7b8492&scope.project.id=8db9b3f8a9f041938ac7b39003a35015&role.id=bec755d6565a41798b09f5f57c1d51dd => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:57.002370 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4767d17a-8766-4672-b2c6-028c1a078970 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:57.021110 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:57.081621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bfa5f3-1889-49b3-8a0e-96504c5a4ae3 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:57.111225 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:57.141619 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554a516-d4ee-455b-96b5-5c19a4fd3d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:57.142566 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554a516-d4ee-455b-96b5-5c19a4fd3d4d None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:57.143021 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554a516-d4ee-455b-96b5-5c19a4fd3d4d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:57.143460 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554a516-d4ee-455b-96b5-5c19a4fd3d4d None admin] PATH_INFO: `/v3/roles/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:57.158304 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554a516-d4ee-455b-96b5-5c19a4fd3d4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:44:57.163233 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-9554a516-d4ee-455b-96b5-5c19a4fd3d4d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:44:57.163233 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:57.163233 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:57.163233 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:57.163233 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:57.163233 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:57.163233 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:57.163233 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:57.163233 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:57.163233 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:57.163233 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:57.163233 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:57.163233 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:44:57.163233 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:44:57.163233 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:44:57.163233 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:44:57.164190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:44:57.164190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:44:57.164190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:44:57.164190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:44:57.164190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:57.164190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:57.164190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:57.164190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:57.164190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:57.164190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:57.164190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:57.164190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:57.164190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:57.164190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:57.164190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:57.164190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:57.165017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:57.165017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:57.165017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:57.165017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:57.165017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:44:57.165017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:44:57.165017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:44:57.165017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:44:57.165017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:44:57.165017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:44:57.165017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:44:57.165017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:57.165017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 288/574] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 31 10:44:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:57.165712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554a516-d4ee-455b-96b5-5c19a4fd3d4d None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:57.183462 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4767d17a-8766-4672-b2c6-028c1a078970 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:57.232351 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5bbcf-c995-4403-934d-d63459e8b62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:57.232969 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5bbcf-c995-4403-934d-d63459e8b62f None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:57.233241 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5bbcf-c995-4403-934d-d63459e8b62f None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:57.233509 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5bbcf-c995-4403-934d-d63459e8b62f None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:57.234205 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5bbcf-c995-4403-934d-d63459e8b62f None admin] RBAC: Adding query filter params (name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:57.234544 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5bbcf-c995-4403-934d-d63459e8b62f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:57.235921 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5bbcf-c995-4403-934d-d63459e8b62f None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:57.246476 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 287/575] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 31 10:44:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:57.247368 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5bbcf-c995-4403-934d-d63459e8b62f None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:57.305992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554a516-d4ee-455b-96b5-5c19a4fd3d4d None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:57.325808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:57.346666 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f666a-939d-4a74-b20b-a329db1cbf50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:57.347444 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f666a-939d-4a74-b20b-a329db1cbf50 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:57.347873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f666a-939d-4a74-b20b-a329db1cbf50 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:57.348337 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f666a-939d-4a74-b20b-a329db1cbf50 None admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:57.349881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f666a-939d-4a74-b20b-a329db1cbf50 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:57.349881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f666a-939d-4a74-b20b-a329db1cbf50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:57.351795 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f666a-939d-4a74-b20b-a329db1cbf50 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:57.372290 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 289/576] 10.222.0.16 () {56 vars in 1133 bytes} [Wed Aug 31 10:44:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:57.373093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f666a-939d-4a74-b20b-a329db1cbf50 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:57.396343 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5bbcf-c995-4403-934d-d63459e8b62f None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:57.420651 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:57.439943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2428f-7604-4003-87f6-4a86527ca653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:57.440545 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2428f-7604-4003-87f6-4a86527ca653 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:57.440814 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2428f-7604-4003-87f6-4a86527ca653 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:57.441081 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2428f-7604-4003-87f6-4a86527ca653 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:57.452428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2428f-7604-4003-87f6-4a86527ca653 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:57.453897 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2428f-7604-4003-87f6-4a86527ca653 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:57.468664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-9bc2428f-7604-4003-87f6-4a86527ca653 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 10:44:57.468664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:57.468664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:57.468664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:57.468664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:57.468664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:57.468664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:57.468664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:57.468664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:57.468664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:57.468664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:57.468664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:57.468664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:44:57.468664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:44:57.468664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:44:57.468664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:44:57.471300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:57.471300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:57.471300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:44:57.471300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:44:57.471300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:44:57.471300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:44:57.471300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:57.471300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:57.471300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:57.471300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:57.471300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:57.471300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:57.471300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:57.471300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:57.471300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:57.471300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:57.471300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:57.471300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:57.472285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:57.472285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:57.472285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:44:57.472285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:44:57.472285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:44:57.472285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:44:57.472285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:44:57.472285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:44:57.472285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 10:44:57.472285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:57.472285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 288/577] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:57.472285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2428f-7604-4003-87f6-4a86527ca653 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:57.498842 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f666a-939d-4a74-b20b-a329db1cbf50 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:57.540996 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03873a1-b9fb-4676-92e8-ad282f287de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:57.541608 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03873a1-b9fb-4676-92e8-ad282f287de8 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:57.541888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03873a1-b9fb-4676-92e8-ad282f287de8 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:57.542161 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03873a1-b9fb-4676-92e8-ad282f287de8 None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:57.542861 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03873a1-b9fb-4676-92e8-ad282f287de8 None admin] RBAC: Adding query filter params (name=service) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:57.543167 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03873a1-b9fb-4676-92e8-ad282f287de8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:57.544700 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03873a1-b9fb-4676-92e8-ad282f287de8 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:57.553251 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 290/578] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 31 10:44:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:57.554109 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03873a1-b9fb-4676-92e8-ad282f287de8 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:57.594565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2428f-7604-4003-87f6-4a86527ca653 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:57.606227 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:57.634167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2d793-4acf-4ed2-840b-2517bd47909a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:57.634864 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2d793-4acf-4ed2-840b-2517bd47909a None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:57.635146 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2d793-4acf-4ed2-840b-2517bd47909a None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:57.635413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2d793-4acf-4ed2-840b-2517bd47909a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:57.647279 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2d793-4acf-4ed2-840b-2517bd47909a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:57.648726 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2d793-4acf-4ed2-840b-2517bd47909a None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:57.664995 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-68c2d793-4acf-4ed2-840b-2517bd47909a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:57.664995 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:57.664995 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:57.664995 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:57.664995 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:57.664995 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:57.664995 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:57.664995 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:57.664995 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:57.664995 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:57.664995 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:57.664995 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:57.664995 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:44:57.664995 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:44:57.664995 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:44:57.664995 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:44:57.665962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:57.665962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:57.665962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:57.665962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:57.665962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:57.665962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:57.665962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:57.665962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:57.665962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:57.665962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:57.665962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:57.665962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:57.665962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:57.665962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:57.665962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:57.665962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:57.665962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:44:57.666858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:44:57.666858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:57.666858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:57.666858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 289/579] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:44:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:57.667089 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2d793-4acf-4ed2-840b-2517bd47909a None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:57.689440 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03873a1-b9fb-4676-92e8-ad282f287de8 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:57.701766 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:57.730920 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894a1fb-fb04-42a3-9bd0-b366530e3df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:57.731893 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894a1fb-fb04-42a3-9bd0-b366530e3df5 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:57.732266 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894a1fb-fb04-42a3-9bd0-b366530e3df5 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:57.732547 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894a1fb-fb04-42a3-9bd0-b366530e3df5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:57.746057 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f894a1fb-fb04-42a3-9bd0-b366530e3df5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:57.747888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f894a1fb-fb04-42a3-9bd0-b366530e3df5 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:57.764907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-f894a1fb-fb04-42a3-9bd0-b366530e3df5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:57.764907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:57.764907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:57.764907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:57.764907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:57.764907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:57.764907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:57.764907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:57.764907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:57.764907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:57.764907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:57.764907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:57.764907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:44:57.764907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:44:57.764907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:44:57.764907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:44:57.765926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:57.765926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:57.765926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:57.765926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:57.765926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:57.765926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:57.765926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:57.765926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:57.765926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:57.765926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:57.765926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:57.765926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:57.765926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:57.765926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:57.765926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:57.765926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:57.765926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:44:57.767214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:44:57.767214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:57.767214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:57.770821 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 291/580] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:44:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:57.772588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894a1fb-fb04-42a3-9bd0-b366530e3df5 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:57.804488 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2d793-4acf-4ed2-840b-2517bd47909a None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:57.847633 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a882cd-d0cc-4214-aefc-919ec5426931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:57.848369 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a882cd-d0cc-4214-aefc-919ec5426931 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:57.848648 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a882cd-d0cc-4214-aefc-919ec5426931 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:57.848920 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a882cd-d0cc-4214-aefc-919ec5426931 None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:57.849688 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a882cd-d0cc-4214-aefc-919ec5426931 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:57.849993 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a882cd-d0cc-4214-aefc-919ec5426931 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:57.851723 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a882cd-d0cc-4214-aefc-919ec5426931 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:57.878885 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 290/581] 10.222.0.16 () {56 vars in 1137 bytes} [Wed Aug 31 10:44:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:57.879072 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:57.879152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a882cd-d0cc-4214-aefc-919ec5426931 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:57.892953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894a1fb-fb04-42a3-9bd0-b366530e3df5 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:57.938124 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867d88d-6dab-4cac-80e8-9807def3fa5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:57.939230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867d88d-6dab-4cac-80e8-9807def3fa5c None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:57.939795 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867d88d-6dab-4cac-80e8-9807def3fa5c None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:57.940374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867d88d-6dab-4cac-80e8-9807def3fa5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:57.953025 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e867d88d-6dab-4cac-80e8-9807def3fa5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:57.954973 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e867d88d-6dab-4cac-80e8-9807def3fa5c None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:57.970990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-e867d88d-6dab-4cac-80e8-9807def3fa5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:57.970990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:57.970990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:57.970990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:57.970990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:57.970990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:57.970990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:57.970990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:57.970990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:57.970990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:57.970990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:57.970990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:57.970990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:44:57.970990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:44:57.970990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:44:57.970990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:44:57.971962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:57.971962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:57.971962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:57.971962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:57.971962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:57.971962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:57.971962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:57.971962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:57.971962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:57.971962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:57.971962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:57.971962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:57.971962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:57.971962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:57.971962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:57.971962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:57.971962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:44:57.972888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:44:57.972888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:57.972888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:57.976731 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 292/582] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:44:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:57.977404 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867d88d-6dab-4cac-80e8-9807def3fa5c None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:57.988024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:57.998665 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a882cd-d0cc-4214-aefc-919ec5426931 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:58.045886 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20946bac-7362-4558-ba83-b949b7f4aa84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:58.046541 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20946bac-7362-4558-ba83-b949b7f4aa84 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:58.046821 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20946bac-7362-4558-ba83-b949b7f4aa84 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:58.047089 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20946bac-7362-4558-ba83-b949b7f4aa84 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:58.047729 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20946bac-7362-4558-ba83-b949b7f4aa84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:58.048028 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20946bac-7362-4558-ba83-b949b7f4aa84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:58.049377 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20946bac-7362-4558-ba83-b949b7f4aa84 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:58.064467 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 291/583] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:58.069569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20946bac-7362-4558-ba83-b949b7f4aa84 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:58.101179 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867d88d-6dab-4cac-80e8-9807def3fa5c None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:58.147794 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67896096-2f33-440f-becd-a1b71ec39204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:58.148420 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67896096-2f33-440f-becd-a1b71ec39204 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:58.148698 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67896096-2f33-440f-becd-a1b71ec39204 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:58.148973 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67896096-2f33-440f-becd-a1b71ec39204 None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:58.149619 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67896096-2f33-440f-becd-a1b71ec39204 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:58.149928 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67896096-2f33-440f-becd-a1b71ec39204 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:58.154898 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67896096-2f33-440f-becd-a1b71ec39204 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:58.163727 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:58.169361 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 293/584] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:58.170253 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67896096-2f33-440f-becd-a1b71ec39204 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:58.211015 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20946bac-7362-4558-ba83-b949b7f4aa84 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:58.250757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae5dae1-5de6-4227-846d-4d90d4e66133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:58.251542 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5dae1-5de6-4227-846d-4d90d4e66133 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:58.251922 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5dae1-5de6-4227-846d-4d90d4e66133 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:58.252293 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5dae1-5de6-4227-846d-4d90d4e66133 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:58.264910 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae5dae1-5de6-4227-846d-4d90d4e66133 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:58.266510 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae5dae1-5de6-4227-846d-4d90d4e66133 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:58.285256 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67896096-2f33-440f-becd-a1b71ec39204 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:58.286873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-0ae5dae1-5de6-4227-846d-4d90d4e66133 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:58.286873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:58.286873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:58.286873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:58.286873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:58.286873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:58.286873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:58.286873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:58.286873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:58.286873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:58.286873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:58.286873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:58.286873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:44:58.286873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:44:58.286873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:44:58.286873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:44:58.288021 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:58.288021 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:58.288021 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:58.288021 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:58.288021 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:58.288021 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:58.288021 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:58.288021 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:58.288021 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:58.288021 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:58.288021 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:58.288021 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:58.288021 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:58.288021 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:58.288021 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:58.288021 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:58.288021 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:44:58.289132 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:44:58.289132 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:58.289132 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:58.289628 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 292/585] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:44:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:58.290452 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae5dae1-5de6-4227-846d-4d90d4e66133 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:58.293692 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:58.333244 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df06fe-eec5-473a-9b98-0ab2a1bbd7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:58.333857 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df06fe-eec5-473a-9b98-0ab2a1bbd7fa None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:58.334129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df06fe-eec5-473a-9b98-0ab2a1bbd7fa None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:58.334444 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df06fe-eec5-473a-9b98-0ab2a1bbd7fa None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:58.335100 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df06fe-eec5-473a-9b98-0ab2a1bbd7fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:58.335404 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df06fe-eec5-473a-9b98-0ab2a1bbd7fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:58.336780 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df06fe-eec5-473a-9b98-0ab2a1bbd7fa None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:58.351290 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 294/586] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:44:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:58.352741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df06fe-eec5-473a-9b98-0ab2a1bbd7fa None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:58.418018 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae5dae1-5de6-4227-846d-4d90d4e66133 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:58.457577 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6315f-70ce-4935-b253-c51797efdcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:58.458232 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6315f-70ce-4935-b253-c51797efdcaf None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:58.458546 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6315f-70ce-4935-b253-c51797efdcaf None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:58.458826 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6315f-70ce-4935-b253-c51797efdcaf None admin] PATH_INFO: `/v3/users/glance` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:58.467464 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c6315f-70ce-4935-b253-c51797efdcaf None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:58.469780 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c6315f-70ce-4935-b253-c51797efdcaf None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:58.473818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df06fe-eec5-473a-9b98-0ab2a1bbd7fa None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:58.478499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:58.485014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-01c6315f-70ce-4935-b253-c51797efdcaf None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 10:44:58.485014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:58.485014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:58.485014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:58.485014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:58.485014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:58.485014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:58.485014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:58.485014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:58.485014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:58.485014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:58.485014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:58.485014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:44:58.485014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:44:58.485014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:44:58.485014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:44:58.486112 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:58.486112 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:58.486112 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:44:58.486112 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:44:58.486112 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:44:58.486112 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:44:58.486112 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:58.486112 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:58.486112 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:58.486112 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:58.486112 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:58.486112 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:58.486112 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:58.486112 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:58.486112 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:58.486112 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:58.486112 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:58.486112 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:58.487093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:58.487093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:58.487093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:44:58.487093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:44:58.487093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:44:58.487093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:44:58.487093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:44:58.487093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:44:58.487093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 10:44:58.487093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:58.490721 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 293/587] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:44:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:58.491867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6315f-70ce-4935-b253-c51797efdcaf None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:58.516187 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b9004c-21fb-4f20-84f2-7b032931655d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:58.516830 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9004c-21fb-4f20-84f2-7b032931655d None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:58.517107 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9004c-21fb-4f20-84f2-7b032931655d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:58.517379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9004c-21fb-4f20-84f2-7b032931655d None admin] PATH_INFO: `/v3/projects/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:58.525702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b9004c-21fb-4f20-84f2-7b032931655d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:58.527227 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b9004c-21fb-4f20-84f2-7b032931655d None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:58.540093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-87b9004c-21fb-4f20-84f2-7b032931655d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:44:58.540093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:58.540093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:58.540093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:58.540093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:58.540093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:58.540093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:58.540093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:58.540093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:58.540093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:58.540093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:58.540093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:58.540093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:44:58.540093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:44:58.540093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:44:58.541225 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:44:58.541225 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:58.541225 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:58.541225 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:58.541225 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:58.541225 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:58.541225 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:58.541225 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:58.541225 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:58.541225 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:58.541225 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:58.541225 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:58.541225 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:58.541225 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:58.541225 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:58.541225 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:58.541225 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:58.542357 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:44:58.542357 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:44:58.542357 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:44:58.542357 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:44:58.542357 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:44:58.542357 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:44:58.542357 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:44:58.542357 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:58.550161 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 295/588] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:44:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:58.550161 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b9004c-21fb-4f20-84f2-7b032931655d None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:58.603950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:58.659348 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6315f-70ce-4935-b253-c51797efdcaf None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:58.665393 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b9004c-21fb-4f20-84f2-7b032931655d None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:58.690562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:58.701555 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641412b-9257-468b-988d-d999a86df1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:58.705744 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641412b-9257-468b-988d-d999a86df1a8 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:58.705744 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641412b-9257-468b-988d-d999a86df1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:58.705744 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641412b-9257-468b-988d-d999a86df1a8 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:58.705744 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e641412b-9257-468b-988d-d999a86df1a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:58.705744 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e641412b-9257-468b-988d-d999a86df1a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:58.705744 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e641412b-9257-468b-988d-d999a86df1a8 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:58.711668 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137e2c2-aa91-4d4c-9f3c-4e1da525a318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:58.712310 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137e2c2-aa91-4d4c-9f3c-4e1da525a318 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:58.712586 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137e2c2-aa91-4d4c-9f3c-4e1da525a318 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:58.712857 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137e2c2-aa91-4d4c-9f3c-4e1da525a318 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:58.720305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 294/589] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:44:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:58.721459 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641412b-9257-468b-988d-d999a86df1a8 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:58.726356 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a137e2c2-aa91-4d4c-9f3c-4e1da525a318 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:58.729375 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a137e2c2-aa91-4d4c-9f3c-4e1da525a318 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:58.750584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-a137e2c2-aa91-4d4c-9f3c-4e1da525a318 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 10:44:58.750584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:58.750584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:58.750584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:58.750584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:58.750584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:58.750584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:58.750584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:58.750584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:58.750584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:58.750584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:58.750584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:58.750584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:44:58.750584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:44:58.750584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:44:58.750584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:44:58.751602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:58.751602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:58.751602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:44:58.751602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:44:58.751602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:44:58.751602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:44:58.751602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:58.751602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:58.751602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:58.751602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:58.751602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:58.751602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:58.751602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:58.751602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:58.751602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:58.751602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:58.751602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:58.751602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:58.752532 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:58.752532 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:58.752532 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:44:58.752532 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:44:58.752532 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:44:58.752532 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:44:58.752532 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:44:58.752532 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:44:58.752532 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 10:44:58.752532 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:58.752532 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 296/590] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 31 10:44:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:58.752532 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137e2c2-aa91-4d4c-9f3c-4e1da525a318 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:58.860221 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641412b-9257-468b-988d-d999a86df1a8 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:58.878945 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137e2c2-aa91-4d4c-9f3c-4e1da525a318 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:58.889462 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:58.909166 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176fc50-f4b9-47a1-8875-f789eeb44d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:58.911508 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176fc50-f4b9-47a1-8875-f789eeb44d8e None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:58.911508 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176fc50-f4b9-47a1-8875-f789eeb44d8e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:58.911508 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176fc50-f4b9-47a1-8875-f789eeb44d8e None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:58.911811 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176fc50-f4b9-47a1-8875-f789eeb44d8e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:58.911904 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176fc50-f4b9-47a1-8875-f789eeb44d8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:58.913347 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176fc50-f4b9-47a1-8875-f789eeb44d8e None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:58.925468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9f103-47ae-429f-aee8-b0e3fd7a9352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:58.926087 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9f103-47ae-429f-aee8-b0e3fd7a9352 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:58.926366 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9f103-47ae-429f-aee8-b0e3fd7a9352 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:58.926669 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9f103-47ae-429f-aee8-b0e3fd7a9352 None admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:58.927355 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9f103-47ae-429f-aee8-b0e3fd7a9352 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:58.927655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9f103-47ae-429f-aee8-b0e3fd7a9352 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:58.929218 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9f103-47ae-429f-aee8-b0e3fd7a9352 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:58.932817 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 295/591] 10.222.0.16 () {56 vars in 1137 bytes} [Wed Aug 31 10:44:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:58.933871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176fc50-f4b9-47a1-8875-f789eeb44d8e None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:58.946639 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 297/592] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:44:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:58.947565 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9f103-47ae-429f-aee8-b0e3fd7a9352 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:59.000846 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:59.052490 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176fc50-f4b9-47a1-8875-f789eeb44d8e None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:59.070580 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9f103-47ae-429f-aee8-b0e3fd7a9352 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:59.096089 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57920a7-dde4-400f-8aa1-e4195622e28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:59.097045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57920a7-dde4-400f-8aa1-e4195622e28e None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:59.097045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57920a7-dde4-400f-8aa1-e4195622e28e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:59.097280 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57920a7-dde4-400f-8aa1-e4195622e28e None admin] PATH_INFO: `/v3/projects/service` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:59.104389 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57920a7-dde4-400f-8aa1-e4195622e28e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:59.105828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57920a7-dde4-400f-8aa1-e4195622e28e None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:59.107112 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fa551-2410-4a58-9b26-632c537d6279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:59.107781 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fa551-2410-4a58-9b26-632c537d6279 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:59.108057 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fa551-2410-4a58-9b26-632c537d6279 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:59.108324 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fa551-2410-4a58-9b26-632c537d6279 None admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:59.109047 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2fa551-2410-4a58-9b26-632c537d6279 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:59.109347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2fa551-2410-4a58-9b26-632c537d6279 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:59.110794 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2fa551-2410-4a58-9b26-632c537d6279 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:59.117696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-e57920a7-dde4-400f-8aa1-e4195622e28e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:44:59.117696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:59.117696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:59.117696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:59.117696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:59.117696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:59.117696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:59.117696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:59.117696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:59.117696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:59.117696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:59.117696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:59.117696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:44:59.117696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:44:59.117696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:44:59.118686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:44:59.118686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:59.118686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:59.118686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:59.118686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:59.118686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:59.118686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:59.118686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:59.118686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:59.118686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:59.118686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:59.118686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:59.118686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:59.118686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:59.118686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:59.118686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:59.118686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:59.119818 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:44:59.119818 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:44:59.119818 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:44:59.119818 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:44:59.119818 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:44:59.119818 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:44:59.119818 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:44:59.119818 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:59.119818 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 296/593] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:44:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:59.120305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57920a7-dde4-400f-8aa1-e4195622e28e None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:59.132110 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 298/594] 10.222.0.16 () {56 vars in 1139 bytes} [Wed Aug 31 10:44:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:59.132884 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fa551-2410-4a58-9b26-632c537d6279 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:59.167929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:59.230199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:59.231043 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57920a7-dde4-400f-8aa1-e4195622e28e None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:59.234067 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fa551-2410-4a58-9b26-632c537d6279 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:59.271367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5671f53-3c94-4dab-8685-e71256df6b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:59.271590 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00f791-9982-4eca-b6d3-39b88a3393e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:59.271935 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5671f53-3c94-4dab-8685-e71256df6b8d None admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:59.272035 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00f791-9982-4eca-b6d3-39b88a3393e6 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:59.272209 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5671f53-3c94-4dab-8685-e71256df6b8d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:59.272309 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00f791-9982-4eca-b6d3-39b88a3393e6 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:59.272546 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5671f53-3c94-4dab-8685-e71256df6b8d None admin] PATH_INFO: `/v3/projects/8db9b3f8a9f041938ac7b39003a35015/users/f11971a594014d86968c3356d11d42f7/roles/bec755d6565a41798b09f5f57c1d51dd` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:59.272804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00f791-9982-4eca-b6d3-39b88a3393e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:59.283545 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00f791-9982-4eca-b6d3-39b88a3393e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:59.284986 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00f791-9982-4eca-b6d3-39b88a3393e6 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:59.287258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5671f53-3c94-4dab-8685-e71256df6b8d None admin] RBAC: Authorizing `identity:create_grant(project_id=8db9b3f8a9f041938ac7b39003a35015, user_id=f11971a594014d86968c3356d11d42f7, role_id=bec755d6565a41798b09f5f57c1d51dd)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:59.288689 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5671f53-3c94-4dab-8685-e71256df6b8d None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:59.298918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-cb00f791-9982-4eca-b6d3-39b88a3393e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:59.298918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:59.298918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:59.298918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:59.298918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:59.298918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:59.298918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:59.298918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:59.298918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:59.298918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:59.298918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:59.298918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:59.298918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:44:59.298918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:44:59.298918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:44:59.298918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:44:59.304565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:59.304565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:59.304565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:59.304565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:59.304565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:59.304565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:59.304565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:59.304565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:59.304565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:59.304565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:59.304565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:59.304565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:59.304565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:59.304565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:59.304565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:59.304565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:59.304565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:44:59.305432 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:44:59.305432 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:59.305432 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:44:59.305432 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 297/595] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:44:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:59.305432 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e2723-8867-4620-ab76-ae51280859c4 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:59.305432 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e2723-8867-4620-ab76-ae51280859c4 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:59.305432 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e2723-8867-4620-ab76-ae51280859c4 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:59.305432 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 298/596] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:44:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:44:59.305432 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e2723-8867-4620-ab76-ae51280859c4 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:59.396398 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 299/597] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 31 10:44:59 2022] PUT /identity/v3/projects/8db9b3f8a9f041938ac7b39003a35015/users/f11971a594014d86968c3356d11d42f7/roles/bec755d6565a41798b09f5f57c1d51dd => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:44:59.397251 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5671f53-3c94-4dab-8685-e71256df6b8d None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:59.399579 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:59.411016 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e2723-8867-4620-ab76-ae51280859c4 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:59.452420 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458c0f5-3909-4a0e-9b50-726b65666a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:59.453049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458c0f5-3909-4a0e-9b50-726b65666a8b None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:59.453326 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458c0f5-3909-4a0e-9b50-726b65666a8b None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:59.453595 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458c0f5-3909-4a0e-9b50-726b65666a8b None admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:59.454326 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458c0f5-3909-4a0e-9b50-726b65666a8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:59.454643 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458c0f5-3909-4a0e-9b50-726b65666a8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:59.456600 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458c0f5-3909-4a0e-9b50-726b65666a8b None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:59.473645 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 299/598] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:44:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:59.473645 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458c0f5-3909-4a0e-9b50-726b65666a8b None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:59.533964 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5671f53-3c94-4dab-8685-e71256df6b8d None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:59.577925 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a1fc7d-fb8f-4afa-8a22-af1d1e3bd27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:59.579265 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1fc7d-fb8f-4afa-8a22-af1d1e3bd27c None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:59.579265 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1fc7d-fb8f-4afa-8a22-af1d1e3bd27c None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:59.579569 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1fc7d-fb8f-4afa-8a22-af1d1e3bd27c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:59.591121 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a1fc7d-fb8f-4afa-8a22-af1d1e3bd27c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:59.592724 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a1fc7d-fb8f-4afa-8a22-af1d1e3bd27c None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:59.596241 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458c0f5-3909-4a0e-9b50-726b65666a8b None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:59.601621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:59.609215 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-f0a1fc7d-fb8f-4afa-8a22-af1d1e3bd27c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:59.609215 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:44:59.609215 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:44:59.609215 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:44:59.609215 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:44:59.609215 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:44:59.609215 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:44:59.609215 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:44:59.609215 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:44:59.609215 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:44:59.609215 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:44:59.609215 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:44:59.609215 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:44:59.609215 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:44:59.609215 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:44:59.609215 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:44:59.615082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:44:59.615082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:44:59.615082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:44:59.615082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:44:59.615082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:44:59.615082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:44:59.615082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:44:59.615082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:44:59.615082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:44:59.615082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:44:59.615082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:44:59.615082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:44:59.615082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:44:59.615082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:44:59.615082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:44:59.615082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:44:59.615082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:44:59.615956 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:44:59.615956 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:44:59.615956 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:44:59.615956 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 300/599] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:44:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:44:59.615956 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273bdc2b-adba-4a1e-8787-a62d16eb9685 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:59.615956 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273bdc2b-adba-4a1e-8787-a62d16eb9685 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:59.615956 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273bdc2b-adba-4a1e-8787-a62d16eb9685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:59.630859 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-273bdc2b-adba-4a1e-8787-a62d16eb9685 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:44:59.632044 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-273bdc2b-adba-4a1e-8787-a62d16eb9685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:44:59.638174 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc92f6-277b-4c66-83bc-bf4927a42031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:59.638822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc92f6-277b-4c66-83bc-bf4927a42031 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:59.639100 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc92f6-277b-4c66-83bc-bf4927a42031 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:59.639370 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc92f6-277b-4c66-83bc-bf4927a42031 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:59.640018 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc92f6-277b-4c66-83bc-bf4927a42031 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:59.640319 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc92f6-277b-4c66-83bc-bf4927a42031 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:59.642182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc92f6-277b-4c66-83bc-bf4927a42031 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:59.659890 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 300/600] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:44:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:59.661237 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc92f6-277b-4c66-83bc-bf4927a42031 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:59.677205 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:59.707673 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 301/601] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:44:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:44:59.714651 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273bdc2b-adba-4a1e-8787-a62d16eb9685 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:59.786255 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc92f6-277b-4c66-83bc-bf4927a42031 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:59.834185 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c634c1-a34c-4fd4-aee1-be58536f43ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:59.835490 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c634c1-a34c-4fd4-aee1-be58536f43ad None admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:59.835881 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c634c1-a34c-4fd4-aee1-be58536f43ad None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:59.836272 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c634c1-a34c-4fd4-aee1-be58536f43ad None admin] PATH_INFO: `/v3/projects/8db9b3f8a9f041938ac7b39003a35015/users/49f9aeb9857b45d68c3baa8fc4acc5fb/roles/bec755d6565a41798b09f5f57c1d51dd` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:59.843984 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273bdc2b-adba-4a1e-8787-a62d16eb9685 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:44:59.858950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c634c1-a34c-4fd4-aee1-be58536f43ad None admin] RBAC: Authorizing `identity:create_grant(project_id=8db9b3f8a9f041938ac7b39003a35015, user_id=49f9aeb9857b45d68c3baa8fc4acc5fb, role_id=bec755d6565a41798b09f5f57c1d51dd)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:59.861036 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c634c1-a34c-4fd4-aee1-be58536f43ad None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:59.888198 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:44:59.894578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad689fa-196a-40d6-bd29-9c9000e6c929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:44:59.895216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad689fa-196a-40d6-bd29-9c9000e6c929 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:44:59.895561 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad689fa-196a-40d6-bd29-9c9000e6c929 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:44:59.895846 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad689fa-196a-40d6-bd29-9c9000e6c929 None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:44:59.896660 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad689fa-196a-40d6-bd29-9c9000e6c929 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:44:59.896973 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad689fa-196a-40d6-bd29-9c9000e6c929 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:44:59.898367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad689fa-196a-40d6-bd29-9c9000e6c929 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:44:59.916057 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 302/602] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:44:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:44:59.916057 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad689fa-196a-40d6-bd29-9c9000e6c929 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:44:59.950694 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 301/603] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 31 10:44:59 2022] PUT /identity/v3/projects/8db9b3f8a9f041938ac7b39003a35015/users/49f9aeb9857b45d68c3baa8fc4acc5fb/roles/bec755d6565a41798b09f5f57c1d51dd => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:44:59.951747 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c634c1-a34c-4fd4-aee1-be58536f43ad None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:00.051454 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:00.068833 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad689fa-196a-40d6-bd29-9c9000e6c929 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:00.105520 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c634c1-a34c-4fd4-aee1-be58536f43ad None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:00.138722 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d80e7-d8cf-4396-a9bc-dd5cdbdd077d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:00.140050 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d80e7-d8cf-4396-a9bc-dd5cdbdd077d None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:00.140447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d80e7-d8cf-4396-a9bc-dd5cdbdd077d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:00.140814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d80e7-d8cf-4396-a9bc-dd5cdbdd077d None admin] PATH_INFO: `/v3/projects/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:00.150228 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d80e7-d8cf-4396-a9bc-dd5cdbdd077d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:00.151319 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f5896-2f0a-4db4-b05a-2a1a77d1a112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:00.151957 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f5896-2f0a-4db4-b05a-2a1a77d1a112 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:00.152051 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d80e7-d8cf-4396-a9bc-dd5cdbdd077d None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:00.152317 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f5896-2f0a-4db4-b05a-2a1a77d1a112 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:00.152602 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f5896-2f0a-4db4-b05a-2a1a77d1a112 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:00.161657 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5f5896-2f0a-4db4-b05a-2a1a77d1a112 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:00.163235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5f5896-2f0a-4db4-b05a-2a1a77d1a112 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:00.174424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-bf0d80e7-d8cf-4396-a9bc-dd5cdbdd077d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:00.174424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:00.174424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:00.174424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:00.174424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:00.174424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:00.174424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:00.174424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:00.174424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:00.174424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:00.174424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:00.174424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:00.174424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:45:00.174424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:45:00.174424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:45:00.175503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:45:00.175503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:00.175503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:00.175503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:00.175503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:00.175503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:00.175503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:00.175503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:00.175503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:00.175503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:00.175503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:00.175503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:00.175503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:00.175503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:00.175503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:00.175503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:00.175503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:00.176480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:45:00.176480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:45:00.176480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:45:00.176480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:45:00.176480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:45:00.176480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:45:00.176480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:00.176480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:00.176480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 303/604] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:44:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:00.176480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d80e7-d8cf-4396-a9bc-dd5cdbdd077d None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:00.180479 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-4f5f5896-2f0a-4db4-b05a-2a1a77d1a112 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:00.180479 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:00.180479 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:00.180479 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:00.180479 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:00.180479 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:00.180479 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:00.180479 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:00.180479 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:00.180479 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:00.180479 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:00.180479 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:00.180479 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:45:00.180479 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:45:00.180479 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:45:00.181503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:45:00.181503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:00.181503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:00.181503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:00.181503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:00.181503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:00.181503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:00.181503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:00.181503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:00.181503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:00.181503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:00.181503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:00.181503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:00.181503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:00.181503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:00.181503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:00.181503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:00.182578 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:45:00.182578 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:45:00.182578 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:45:00.182578 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:45:00.182578 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:45:00.182578 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:45:00.182578 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:00.182578 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:00.183088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 302/605] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:44:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:00.186445 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f5896-2f0a-4db4-b05a-2a1a77d1a112 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:00.245483 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:00.298325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f5896-2f0a-4db4-b05a-2a1a77d1a112 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:00.315771 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d80e7-d8cf-4396-a9bc-dd5cdbdd077d None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:00.327866 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:00.337990 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40d7e5f-f3b7-4704-b647-86c113191e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:00.339057 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d7e5f-f3b7-4704-b647-86c113191e4d None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:00.339399 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d7e5f-f3b7-4704-b647-86c113191e4d None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:00.339752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d7e5f-f3b7-4704-b647-86c113191e4d None admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:00.341779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40d7e5f-f3b7-4704-b647-86c113191e4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:00.342158 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40d7e5f-f3b7-4704-b647-86c113191e4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:00.343813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40d7e5f-f3b7-4704-b647-86c113191e4d None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:00.364902 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 303/606] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:45:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:00.365861 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c1bd8-0d9d-4b2e-b3f0-0996a92ce0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:00.366022 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40d7e5f-f3b7-4704-b647-86c113191e4d None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:00.367261 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c1bd8-0d9d-4b2e-b3f0-0996a92ce0af None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:00.367918 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c1bd8-0d9d-4b2e-b3f0-0996a92ce0af None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:00.368279 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c1bd8-0d9d-4b2e-b3f0-0996a92ce0af None admin] PATH_INFO: `/v3/roles/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:00.383761 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28c1bd8-0d9d-4b2e-b3f0-0996a92ce0af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:45:00.393496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-f28c1bd8-0d9d-4b2e-b3f0-0996a92ce0af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:45:00.393496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:00.393496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:00.393496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:00.393496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:00.393496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:00.393496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:00.393496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:00.393496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:00.393496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:00.393496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:00.393496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:00.393496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:45:00.393496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:45:00.393496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:45:00.393496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:45:00.394702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:45:00.394702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:45:00.394702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:45:00.394702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:45:00.394702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:00.394702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:00.394702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:00.394702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:00.394702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:00.394702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:00.394702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:00.394702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:00.394702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:00.394702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:00.394702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:00.394702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:00.395724 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:00.395724 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:00.395724 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:00.395724 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:00.395724 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:45:00.395724 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:45:00.395724 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:45:00.395724 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:45:00.395724 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:45:00.395724 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:45:00.395724 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:45:00.395724 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:00.399799 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 304/607] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 31 10:45:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:00.399799 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c1bd8-0d9d-4b2e-b3f0-0996a92ce0af None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:00.497882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40d7e5f-f3b7-4704-b647-86c113191e4d None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:00.543176 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9d3b6-8c30-4a42-9227-e5ff4143d35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:00.544080 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9d3b6-8c30-4a42-9227-e5ff4143d35e None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:00.544417 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9d3b6-8c30-4a42-9227-e5ff4143d35e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:00.545025 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9d3b6-8c30-4a42-9227-e5ff4143d35e None admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:00.545767 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f9d3b6-8c30-4a42-9227-e5ff4143d35e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:00.546110 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f9d3b6-8c30-4a42-9227-e5ff4143d35e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:00.546960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c1bd8-0d9d-4b2e-b3f0-0996a92ce0af None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:00.547606 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f9d3b6-8c30-4a42-9227-e5ff4143d35e None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:00.561281 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:00.574790 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 304/608] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:45:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:00.574790 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9d3b6-8c30-4a42-9227-e5ff4143d35e None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:00.588615 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fc458-ee78-4996-9117-28a99254057a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:00.589699 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fc458-ee78-4996-9117-28a99254057a None admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:00.590043 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fc458-ee78-4996-9117-28a99254057a None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:00.590433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fc458-ee78-4996-9117-28a99254057a None admin] PATH_INFO: `/v3/projects/8db9b3f8a9f041938ac7b39003a35015/users/f2e72c8e7b444e978bf99c6404febcd2/roles/bec755d6565a41798b09f5f57c1d51dd` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:00.610642 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25fc458-ee78-4996-9117-28a99254057a None admin] RBAC: Authorizing `identity:create_grant(project_id=8db9b3f8a9f041938ac7b39003a35015, user_id=f2e72c8e7b444e978bf99c6404febcd2, role_id=bec755d6565a41798b09f5f57c1d51dd)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:00.612204 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25fc458-ee78-4996-9117-28a99254057a None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:00.629080 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:00.692290 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9d3b6-8c30-4a42-9227-e5ff4143d35e None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:00.718822 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 305/609] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 31 10:45:00 2022] PUT /identity/v3/projects/8db9b3f8a9f041938ac7b39003a35015/users/f2e72c8e7b444e978bf99c6404febcd2/roles/bec755d6565a41798b09f5f57c1d51dd => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:45:00.720634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fc458-ee78-4996-9117-28a99254057a None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:00.769925 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83514ccb-7306-402e-b436-3b0b0a2f023e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:00.770965 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83514ccb-7306-402e-b436-3b0b0a2f023e None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:00.771456 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83514ccb-7306-402e-b436-3b0b0a2f023e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:00.771956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83514ccb-7306-402e-b436-3b0b0a2f023e None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:00.772898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83514ccb-7306-402e-b436-3b0b0a2f023e None admin] RBAC: Adding query filter params (name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:00.773486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83514ccb-7306-402e-b436-3b0b0a2f023e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:00.775532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83514ccb-7306-402e-b436-3b0b0a2f023e None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:00.796583 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 305/610] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 31 10:45:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:00.802331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83514ccb-7306-402e-b436-3b0b0a2f023e None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:00.925680 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fc458-ee78-4996-9117-28a99254057a None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:00.975893 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83514ccb-7306-402e-b436-3b0b0a2f023e None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:00.979252 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:00.987333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78d671-48f2-40b5-9c4e-9d2a9499649b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:00.988011 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78d671-48f2-40b5-9c4e-9d2a9499649b None admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:00.988348 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78d671-48f2-40b5-9c4e-9d2a9499649b None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:00.988883 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78d671-48f2-40b5-9c4e-9d2a9499649b None admin] PATH_INFO: `/v3/projects/8db9b3f8a9f041938ac7b39003a35015/users/1a00b7f45b2e47b8a01048c36d00f742/roles/bec755d6565a41798b09f5f57c1d51dd` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:01.006885 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa78d671-48f2-40b5-9c4e-9d2a9499649b None admin] RBAC: Authorizing `identity:create_grant(project_id=8db9b3f8a9f041938ac7b39003a35015, user_id=1a00b7f45b2e47b8a01048c36d00f742, role_id=bec755d6565a41798b09f5f57c1d51dd)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:01.008397 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa78d671-48f2-40b5-9c4e-9d2a9499649b None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:01.031796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b9919-104b-46ab-985c-9051daefbaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:01.032491 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b9919-104b-46ab-985c-9051daefbaf0 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:01.032800 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b9919-104b-46ab-985c-9051daefbaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:01.033150 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b9919-104b-46ab-985c-9051daefbaf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:01.045477 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b9919-104b-46ab-985c-9051daefbaf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:01.047045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b9919-104b-46ab-985c-9051daefbaf0 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:01.065014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-ad3b9919-104b-46ab-985c-9051daefbaf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:01.065014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:01.065014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:01.065014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:01.065014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:01.065014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:01.065014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:01.065014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:01.065014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:01.065014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:01.065014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:01.065014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:01.065014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:01.065014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:01.065014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:01.065014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:01.066183 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:01.066183 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:01.066183 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:01.066183 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:01.066183 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:01.066183 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:01.066183 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:01.066183 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:01.066183 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:01.066183 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:01.066183 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:01.066183 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:01.066183 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:01.066183 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:01.066183 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:01.066183 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:01.066183 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:01.067308 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:01.067308 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:01.067308 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:01.067308 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 306/611] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:01.072071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b9919-104b-46ab-985c-9051daefbaf0 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:01.081911 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 306/612] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 31 10:45:00 2022] PUT /identity/v3/projects/8db9b3f8a9f041938ac7b39003a35015/users/1a00b7f45b2e47b8a01048c36d00f742/roles/bec755d6565a41798b09f5f57c1d51dd => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:45:01.211998 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b9919-104b-46ab-985c-9051daefbaf0 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:01.271370 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd69c81f-8603-4b3f-96f6-2dd18227eddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:01.272006 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69c81f-8603-4b3f-96f6-2dd18227eddc None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:01.272290 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69c81f-8603-4b3f-96f6-2dd18227eddc None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:01.272563 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69c81f-8603-4b3f-96f6-2dd18227eddc None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:01.273234 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd69c81f-8603-4b3f-96f6-2dd18227eddc None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:01.273541 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd69c81f-8603-4b3f-96f6-2dd18227eddc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:01.275083 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd69c81f-8603-4b3f-96f6-2dd18227eddc None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:01.289910 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 307/613] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:01.295982 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78d671-48f2-40b5-9c4e-9d2a9499649b None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:01.319074 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:01.428670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78d671-48f2-40b5-9c4e-9d2a9499649b None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:01.476151 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ddbed-04f7-4734-a10e-0c181699f8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:01.476719 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ddbed-04f7-4734-a10e-0c181699f8b5 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:01.476995 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ddbed-04f7-4734-a10e-0c181699f8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:01.477259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ddbed-04f7-4734-a10e-0c181699f8b5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:01.487964 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ddbed-04f7-4734-a10e-0c181699f8b5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:01.489470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ddbed-04f7-4734-a10e-0c181699f8b5 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:01.502640 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-0c4ddbed-04f7-4734-a10e-0c181699f8b5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 10:45:01.502640 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:01.502640 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:01.502640 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:01.502640 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:01.502640 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:01.502640 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:01.502640 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:01.502640 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:01.502640 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:01.502640 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:01.502640 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:01.502640 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:45:01.502640 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:45:01.502640 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:45:01.502640 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:45:01.503621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:01.503621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:01.503621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:45:01.503621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:01.503621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:45:01.503621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:01.503621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:01.503621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:01.503621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:01.503621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:01.503621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:01.503621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:01.503621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:01.503621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:01.503621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:01.503621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:01.503621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:01.503621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:01.505164 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:01.505164 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:01.505164 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:45:01.505164 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:45:01.505164 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:45:01.505164 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:45:01.505164 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:45:01.505164 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:45:01.505164 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 10:45:01.505164 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:01.505164 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 307/614] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:45:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:01.509341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd69c81f-8603-4b3f-96f6-2dd18227eddc None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:01.538657 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d5446-3d90-4d29-9f49-2c844a990ee0 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:01.538852 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:01.538976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d5446-3d90-4d29-9f49-2c844a990ee0 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:01.539262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d5446-3d90-4d29-9f49-2c844a990ee0 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:01.540714 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 308/615] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:01.547119 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0471691-df8c-41bb-968f-533544ccab84 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:01.552182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0471691-df8c-41bb-968f-533544ccab84 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:01.552182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0471691-df8c-41bb-968f-533544ccab84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:01.568282 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-f0471691-df8c-41bb-968f-533544ccab84 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:01.569540 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-f0471691-df8c-41bb-968f-533544ccab84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:01.648788 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 309/616] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:45:01.650828 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:01.689807 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd69c81f-8603-4b3f-96f6-2dd18227eddc None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:01.734749 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec1fdf-0a5e-42ae-b917-56f19d518892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:01.735292 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec1fdf-0a5e-42ae-b917-56f19d518892 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:01.735573 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec1fdf-0a5e-42ae-b917-56f19d518892 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:01.735845 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec1fdf-0a5e-42ae-b917-56f19d518892 None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:01.736589 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec1fdf-0a5e-42ae-b917-56f19d518892 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:01.736894 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec1fdf-0a5e-42ae-b917-56f19d518892 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:01.738331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec1fdf-0a5e-42ae-b917-56f19d518892 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:01.766368 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 308/617] 10.222.0.16 () {56 vars in 1135 bytes} [Wed Aug 31 10:45:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:01.772586 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0471691-df8c-41bb-968f-533544ccab84 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:01.942551 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0471691-df8c-41bb-968f-533544ccab84 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:01.978964 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1125c1ef-ecec-40eb-8fab-4517dda4ede5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:01.979602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125c1ef-ecec-40eb-8fab-4517dda4ede5 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:01.979876 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125c1ef-ecec-40eb-8fab-4517dda4ede5 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:01.980144 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125c1ef-ecec-40eb-8fab-4517dda4ede5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:01.991222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1125c1ef-ecec-40eb-8fab-4517dda4ede5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:01.992610 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1125c1ef-ecec-40eb-8fab-4517dda4ede5 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:02.008424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-1125c1ef-ecec-40eb-8fab-4517dda4ede5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:02.008424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:02.008424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:02.008424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:02.008424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:02.008424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:02.008424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:02.008424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:02.008424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:02.008424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:02.008424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:02.008424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:02.008424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:02.008424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:02.008424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:02.008424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:02.009578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:02.009578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:02.009578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:02.009578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:02.009578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:02.009578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:02.009578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:02.009578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:02.009578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:02.009578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:02.009578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:02.009578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:02.009578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:02.009578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:02.009578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:02.009578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:02.009578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:02.010515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:02.010515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:02.010515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:02.010515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 310/618] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:02.016014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec1fdf-0a5e-42ae-b917-56f19d518892 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:02.071305 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1125c1ef-ecec-40eb-8fab-4517dda4ede5 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:02.121757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:02.149160 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec1fdf-0a5e-42ae-b917-56f19d518892 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:02.173665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:02.219943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1161c-22b4-4741-92cb-825b52741d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:02.222723 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1161c-22b4-4741-92cb-825b52741d4f None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:02.222999 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1161c-22b4-4741-92cb-825b52741d4f None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:02.223268 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1161c-22b4-4741-92cb-825b52741d4f None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:02.223915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a1161c-22b4-4741-92cb-825b52741d4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:02.224217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a1161c-22b4-4741-92cb-825b52741d4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:02.225577 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a1161c-22b4-4741-92cb-825b52741d4f None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:02.250022 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1125c1ef-ecec-40eb-8fab-4517dda4ede5 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:02.251857 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 309/619] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:02.256919 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1161c-22b4-4741-92cb-825b52741d4f None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:02.308193 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f19ca3b-5399-4967-8519-003187410a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:02.309351 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19ca3b-5399-4967-8519-003187410a47 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:02.309626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19ca3b-5399-4967-8519-003187410a47 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:02.310160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19ca3b-5399-4967-8519-003187410a47 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:02.325159 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f19ca3b-5399-4967-8519-003187410a47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:45:02.333137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-7f19ca3b-5399-4967-8519-003187410a47 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:45:02.333137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:02.333137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:02.333137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:02.333137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:02.333137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:02.333137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:02.333137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:02.333137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:02.333137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:02.333137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:02.333137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:02.333137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:45:02.333137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:45:02.333137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:45:02.333137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:45:02.334382 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:45:02.334382 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:45:02.334382 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:45:02.334382 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:45:02.334382 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:02.334382 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:02.334382 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:02.334382 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:02.334382 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:02.334382 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:02.334382 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:02.334382 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:02.334382 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:02.334382 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:02.334382 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:02.334382 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:02.335452 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:02.335452 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:02.335452 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:02.335452 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:02.335452 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:45:02.335452 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:45:02.335452 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:45:02.335452 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:45:02.335452 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:45:02.335452 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:45:02.335452 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:45:02.335452 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:02.335452 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 311/620] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 31 10:45:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:02.336322 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23139c70-7c32-4548-ac7a-3257a5a8a62b None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:02.336456 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23139c70-7c32-4548-ac7a-3257a5a8a62b None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:02.338425 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23139c70-7c32-4548-ac7a-3257a5a8a62b None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:02.338877 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 312/621] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:02.343996 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23139c70-7c32-4548-ac7a-3257a5a8a62b None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:02.403946 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1161c-22b4-4741-92cb-825b52741d4f None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:02.451370 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece39aa-9808-488f-a8fc-c1d22d524c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:02.452109 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece39aa-9808-488f-a8fc-c1d22d524c71 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:02.452452 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece39aa-9808-488f-a8fc-c1d22d524c71 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:02.452785 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece39aa-9808-488f-a8fc-c1d22d524c71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:02.456142 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:02.465047 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece39aa-9808-488f-a8fc-c1d22d524c71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:02.468249 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece39aa-9808-488f-a8fc-c1d22d524c71 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:02.471778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23139c70-7c32-4548-ac7a-3257a5a8a62b None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:02.483748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-eece39aa-9808-488f-a8fc-c1d22d524c71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:02.483748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:02.483748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:02.483748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:02.483748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:02.483748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:02.483748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:02.483748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:02.483748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:02.483748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:02.483748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:02.483748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:02.483748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:45:02.483748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:45:02.483748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:45:02.484692 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:45:02.484692 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:02.484692 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:02.484692 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:02.484692 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:02.484692 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:02.484692 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:02.484692 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:02.484692 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:02.484692 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:02.484692 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:02.484692 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:02.484692 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:02.484692 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:02.484692 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:02.484692 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:02.484692 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:02.485546 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:45:02.485546 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:45:02.485546 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:45:02.485546 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:45:02.485546 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:45:02.485546 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:45:02.485546 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:02.485546 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:02.486601 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:02.489262 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 310/622] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:45:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:02.490814 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb532ae-c3e3-4542-8396-61c0155183fc None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:02.491226 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb532ae-c3e3-4542-8396-61c0155183fc None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:02.491644 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb532ae-c3e3-4542-8396-61c0155183fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:02.516568 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-ccb532ae-c3e3-4542-8396-61c0155183fc None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:02.516837 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede58590-0508-4e81-a4e3-f6696eceaf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:02.517549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede58590-0508-4e81-a4e3-f6696eceaf95 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:02.517930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede58590-0508-4e81-a4e3-f6696eceaf95 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:02.518327 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede58590-0508-4e81-a4e3-f6696eceaf95 None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:02.518706 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-ccb532ae-c3e3-4542-8396-61c0155183fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:02.519337 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede58590-0508-4e81-a4e3-f6696eceaf95 None admin] RBAC: Adding query filter params (name=service) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:02.519750 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede58590-0508-4e81-a4e3-f6696eceaf95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:02.521473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede58590-0508-4e81-a4e3-f6696eceaf95 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:02.532591 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 313/623] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 31 10:45:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:02.533656 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede58590-0508-4e81-a4e3-f6696eceaf95 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:02.622951 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 311/624] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:45:02.623797 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb532ae-c3e3-4542-8396-61c0155183fc None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:02.743103 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede58590-0508-4e81-a4e3-f6696eceaf95 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:02.748216 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb532ae-c3e3-4542-8396-61c0155183fc None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:02.776036 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:02.791163 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752fc32-a7a2-4746-a890-e8b7d02af300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:02.793093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752fc32-a7a2-4746-a890-e8b7d02af300 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:02.793093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752fc32-a7a2-4746-a890-e8b7d02af300 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:02.793093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752fc32-a7a2-4746-a890-e8b7d02af300 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:02.805194 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c752fc32-a7a2-4746-a890-e8b7d02af300 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:02.806739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c752fc32-a7a2-4746-a890-e8b7d02af300 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:02.826307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-c752fc32-a7a2-4746-a890-e8b7d02af300 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:02.826307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:02.826307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:02.826307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:02.826307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:02.826307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:02.826307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:02.826307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:02.826307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:02.826307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:02.826307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:02.826307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:02.826307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:02.826307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:02.826307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:02.826307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:02.827307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:02.827307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:02.827307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:02.827307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:02.827307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:02.827307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:02.827307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:02.827307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:02.827307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:02.827307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:02.827307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:02.827307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:02.827307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:02.827307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:02.827307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:02.827307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:02.827307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:02.828160 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:02.828160 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:02.828160 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:02.828752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 312/625] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:02.835353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752fc32-a7a2-4746-a890-e8b7d02af300 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:02.844813 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246fec09-09d5-4119-80d4-852a6f28c9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:02.847027 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fec09-09d5-4119-80d4-852a6f28c9cf None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:02.847534 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fec09-09d5-4119-80d4-852a6f28c9cf None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:02.847998 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fec09-09d5-4119-80d4-852a6f28c9cf None admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:02.853464 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246fec09-09d5-4119-80d4-852a6f28c9cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:02.853991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246fec09-09d5-4119-80d4-852a6f28c9cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:02.855912 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246fec09-09d5-4119-80d4-852a6f28c9cf None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:02.883510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 314/626] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:45:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:02.889383 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246fec09-09d5-4119-80d4-852a6f28c9cf None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:02.941829 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:03.013422 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752fc32-a7a2-4746-a890-e8b7d02af300 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:03.027489 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246fec09-09d5-4119-80d4-852a6f28c9cf None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:03.069501 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414cf98-7743-4232-8517-89c21ae38d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:03.070241 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414cf98-7743-4232-8517-89c21ae38d19 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:03.070541 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414cf98-7743-4232-8517-89c21ae38d19 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:03.070811 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414cf98-7743-4232-8517-89c21ae38d19 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:03.071456 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414cf98-7743-4232-8517-89c21ae38d19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:03.071761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414cf98-7743-4232-8517-89c21ae38d19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:03.072220 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65db6dd-fb95-4740-998c-9b6924ff80e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:03.072881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65db6dd-fb95-4740-998c-9b6924ff80e4 None admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:03.073119 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414cf98-7743-4232-8517-89c21ae38d19 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:03.073216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65db6dd-fb95-4740-998c-9b6924ff80e4 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:03.073422 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65db6dd-fb95-4740-998c-9b6924ff80e4 None admin] PATH_INFO: `/v3/projects/8db9b3f8a9f041938ac7b39003a35015/users/e8abc7c439a04622896b34ed0a7b8492/roles/bec755d6565a41798b09f5f57c1d51dd` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:03.091846 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 313/627] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:03.091846 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414cf98-7743-4232-8517-89c21ae38d19 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:03.099342 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65db6dd-fb95-4740-998c-9b6924ff80e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=8db9b3f8a9f041938ac7b39003a35015, user_id=e8abc7c439a04622896b34ed0a7b8492, role_id=bec755d6565a41798b09f5f57c1d51dd)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:03.100817 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65db6dd-fb95-4740-998c-9b6924ff80e4 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:03.127233 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:03.182983 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 315/628] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 31 10:45:02 2022] PUT /identity/v3/projects/8db9b3f8a9f041938ac7b39003a35015/users/e8abc7c439a04622896b34ed0a7b8492/roles/bec755d6565a41798b09f5f57c1d51dd => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 10:45:03.185215 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f986700-9e78-49c1-8a1a-07d5080165da None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:03.185573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f986700-9e78-49c1-8a1a-07d5080165da None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:03.185914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f986700-9e78-49c1-8a1a-07d5080165da None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:03.187489 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 316/629] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:03.192519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f986700-9e78-49c1-8a1a-07d5080165da None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:03.203859 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414cf98-7743-4232-8517-89c21ae38d19 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:03.251991 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923d47e-070a-4a4b-a41e-138fd6ab1f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:03.253420 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923d47e-070a-4a4b-a41e-138fd6ab1f2a None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:03.253893 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923d47e-070a-4a4b-a41e-138fd6ab1f2a None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:03.254326 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923d47e-070a-4a4b-a41e-138fd6ab1f2a None admin] PATH_INFO: `/v3/roles/service` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:03.268734 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923d47e-070a-4a4b-a41e-138fd6ab1f2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:45:03.274454 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-6923d47e-070a-4a4b-a41e-138fd6ab1f2a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:45:03.274454 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:03.274454 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:03.274454 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:03.274454 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:03.274454 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:03.274454 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:03.274454 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:03.274454 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:03.274454 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:03.274454 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:03.274454 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:03.274454 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:45:03.274454 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:45:03.274454 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:45:03.274454 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:45:03.275522 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:45:03.275522 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:45:03.275522 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:45:03.275522 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:45:03.275522 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:03.275522 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:03.275522 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:03.275522 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:03.275522 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:03.275522 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:03.275522 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:03.275522 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:03.275522 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:03.275522 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:03.275522 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:03.275522 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:03.276333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:03.276333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:03.276333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:03.276333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:03.276333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:45:03.276333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:45:03.276333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:45:03.276333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:45:03.276333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:45:03.276333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:45:03.276333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:45:03.276333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:03.279817 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 314/630] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 31 10:45:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:03.281370 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166a1e1-8566-4bd0-bfee-5c0d93510087 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:03.281893 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166a1e1-8566-4bd0-bfee-5c0d93510087 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:03.282323 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166a1e1-8566-4bd0-bfee-5c0d93510087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:03.303498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-0166a1e1-8566-4bd0-bfee-5c0d93510087 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:03.305065 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-0166a1e1-8566-4bd0-bfee-5c0d93510087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:03.319782 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:03.345145 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f986700-9e78-49c1-8a1a-07d5080165da None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:03.398231 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 315/631] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:45:03.399181 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166a1e1-8566-4bd0-bfee-5c0d93510087 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:03.400873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b3172d-7da9-4f29-a5c0-3c68906cf83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:03.401519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3172d-7da9-4f29-a5c0-3c68906cf83c None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:03.401796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3172d-7da9-4f29-a5c0-3c68906cf83c None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:03.402068 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3172d-7da9-4f29-a5c0-3c68906cf83c None admin] PATH_INFO: `/v3/users/nova` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:03.411033 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b3172d-7da9-4f29-a5c0-3c68906cf83c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:03.412622 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b3172d-7da9-4f29-a5c0-3c68906cf83c None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:03.427042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-31b3172d-7da9-4f29-a5c0-3c68906cf83c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:45:03.427042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:03.427042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:03.427042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:03.427042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:03.427042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:03.427042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:03.427042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:03.427042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:03.427042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:03.427042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:03.427042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:03.427042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:45:03.427042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:45:03.427042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:45:03.427042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:45:03.428058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:03.428058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:03.428058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:45:03.428058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:03.428058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:45:03.428058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:03.428058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:03.428058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:03.428058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:03.428058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:03.428058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:03.428058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:03.428058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:03.428058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:03.428058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:03.428058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:03.428058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:03.428058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:03.429005 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:03.429005 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:03.429005 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:45:03.429005 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:45:03.429005 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:45:03.429005 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:45:03.429005 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:45:03.429005 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:45:03.429005 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:45:03.429005 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:03.429005 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 317/632] 10.222.0.16 () {56 vars in 1093 bytes} [Wed Aug 31 10:45:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:03.434170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b3172d-7da9-4f29-a5c0-3c68906cf83c None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:03.442241 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:03.562499 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166a1e1-8566-4bd0-bfee-5c0d93510087 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:03.600740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a398bd8-af5d-4374-8f1a-60bbae5f4d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:03.601711 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a398bd8-af5d-4374-8f1a-60bbae5f4d0b None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:03.602168 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a398bd8-af5d-4374-8f1a-60bbae5f4d0b None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:03.602650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a398bd8-af5d-4374-8f1a-60bbae5f4d0b None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:03.603532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a398bd8-af5d-4374-8f1a-60bbae5f4d0b None admin] RBAC: Adding query filter params (name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:03.604055 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a398bd8-af5d-4374-8f1a-60bbae5f4d0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:03.605637 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a398bd8-af5d-4374-8f1a-60bbae5f4d0b None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:03.607819 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b3172d-7da9-4f29-a5c0-3c68906cf83c None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:03.616496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 316/633] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 31 10:45:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:03.618069 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf4e66-4c0f-45bd-8285-31681aed603b None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:03.618580 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf4e66-4c0f-45bd-8285-31681aed603b None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:03.619023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf4e66-4c0f-45bd-8285-31681aed603b None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:03.620677 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 317/634] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:03.622931 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cf4e66-4c0f-45bd-8285-31681aed603b None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:03.658359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206095f-57b5-401f-a3df-b04d721e7713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:03.659524 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206095f-57b5-401f-a3df-b04d721e7713 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:03.659999 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206095f-57b5-401f-a3df-b04d721e7713 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:03.660736 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206095f-57b5-401f-a3df-b04d721e7713 None admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:03.662659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a206095f-57b5-401f-a3df-b04d721e7713 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:03.663183 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a206095f-57b5-401f-a3df-b04d721e7713 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:03.664877 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a206095f-57b5-401f-a3df-b04d721e7713 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:03.691149 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 318/635] 10.222.0.16 () {56 vars in 1133 bytes} [Wed Aug 31 10:45:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:03.693559 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7812a7a-3c56-485e-a400-48ac5da0acac None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:03.694124 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7812a7a-3c56-485e-a400-48ac5da0acac None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:03.694593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7812a7a-3c56-485e-a400-48ac5da0acac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:03.703805 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:03.722483 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-e7812a7a-3c56-485e-a400-48ac5da0acac None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:03.723718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-e7812a7a-3c56-485e-a400-48ac5da0acac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:03.728330 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:03.742842 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cf4e66-4c0f-45bd-8285-31681aed603b None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:03.779460 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae24ec-1ffc-457a-ac5b-7eea8d059835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:03.780283 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae24ec-1ffc-457a-ac5b-7eea8d059835 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:03.781252 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae24ec-1ffc-457a-ac5b-7eea8d059835 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:03.781533 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae24ec-1ffc-457a-ac5b-7eea8d059835 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:03.793673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae24ec-1ffc-457a-ac5b-7eea8d059835 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:03.795355 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae24ec-1ffc-457a-ac5b-7eea8d059835 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:03.806449 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 319/636] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:45:03.806449 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7812a7a-3c56-485e-a400-48ac5da0acac None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:03.811635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-d3ae24ec-1ffc-457a-ac5b-7eea8d059835 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:03.811635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:03.811635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:03.811635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:03.811635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:03.811635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:03.811635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:03.811635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:03.811635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:03.811635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:03.811635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:03.811635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:03.811635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:03.811635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:03.811635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:03.811635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:03.812764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:03.812764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:03.812764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:03.812764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:03.812764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:03.812764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:03.812764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:03.812764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:03.812764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:03.812764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:03.812764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:03.812764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:03.812764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:03.812764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:03.812764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:03.812764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:03.812764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:03.813682 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:03.813682 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:03.813682 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:03.817565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 318/637] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:03.818766 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae24ec-1ffc-457a-ac5b-7eea8d059835 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:03.941218 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7812a7a-3c56-485e-a400-48ac5da0acac None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:03.947910 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae24ec-1ffc-457a-ac5b-7eea8d059835 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:03.974309 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:03.980240 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602717d-0d31-4644-8ea7-ef69116aeab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:03.981605 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602717d-0d31-4644-8ea7-ef69116aeab2 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:03.982027 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602717d-0d31-4644-8ea7-ef69116aeab2 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:03.982463 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602717d-0d31-4644-8ea7-ef69116aeab2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:03.986800 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e684a87-caa6-49ca-b240-d55beaad9649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:03.988864 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e684a87-caa6-49ca-b240-d55beaad9649 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:03.988864 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e684a87-caa6-49ca-b240-d55beaad9649 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:03.988864 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e684a87-caa6-49ca-b240-d55beaad9649 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:03.995846 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:04.000779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e684a87-caa6-49ca-b240-d55beaad9649 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:45:04.005739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-7e684a87-caa6-49ca-b240-d55beaad9649 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:45:04.005739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:04.005739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:04.005739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:04.005739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:04.005739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:04.005739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:04.005739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:04.005739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:04.005739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:04.005739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:04.005739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:04.005739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:45:04.005739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:45:04.005739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:45:04.005739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:45:04.006867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:45:04.006867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:45:04.006867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:45:04.006867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:45:04.006867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:04.006867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:04.006867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:04.006867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:04.006867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:04.006867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:04.006867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:04.006867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:04.006867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:04.006867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:04.006867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:04.006867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:04.007711 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:04.007711 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:04.007711 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:04.007711 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:04.007711 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:45:04.007711 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:45:04.007711 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:45:04.007711 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:45:04.007711 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:45:04.007711 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:45:04.007711 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:45:04.007711 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:04.007711 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 319/638] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 31 10:45:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:04.008390 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e684a87-caa6-49ca-b240-d55beaad9649 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:04.064049 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602717d-0d31-4644-8ea7-ef69116aeab2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:04.065992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602717d-0d31-4644-8ea7-ef69116aeab2 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:04.126607 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e684a87-caa6-49ca-b240-d55beaad9649 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:04.163054 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec872b-ac08-4fd4-873b-8610d6ef6be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:04.163667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec872b-ac08-4fd4-873b-8610d6ef6be5 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:04.164039 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec872b-ac08-4fd4-873b-8610d6ef6be5 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:04.164314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec872b-ac08-4fd4-873b-8610d6ef6be5 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:04.165326 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec872b-ac08-4fd4-873b-8610d6ef6be5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:04.165623 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec872b-ac08-4fd4-873b-8610d6ef6be5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:04.167333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec872b-ac08-4fd4-873b-8610d6ef6be5 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:04.180973 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 320/639] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:04.182135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec872b-ac08-4fd4-873b-8610d6ef6be5 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:04.205688 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 320/640] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:45:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:45:04.205688 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602717d-0d31-4644-8ea7-ef69116aeab2 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:04.233948 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:04.256529 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:04.301156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec872b-ac08-4fd4-873b-8610d6ef6be5 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:04.315523 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602717d-0d31-4644-8ea7-ef69116aeab2 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:04.343047 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71535bff-c2d3-4830-9dc0-dd41cf434d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:04.343676 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71535bff-c2d3-4830-9dc0-dd41cf434d94 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:04.343952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71535bff-c2d3-4830-9dc0-dd41cf434d94 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:04.344224 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71535bff-c2d3-4830-9dc0-dd41cf434d94 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:04.344884 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71535bff-c2d3-4830-9dc0-dd41cf434d94 None admin] RBAC: Adding query filter params (name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:04.345200 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71535bff-c2d3-4830-9dc0-dd41cf434d94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:04.346585 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71535bff-c2d3-4830-9dc0-dd41cf434d94 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:04.355319 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 321/641] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 31 10:45:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:04.356603 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ad77b-2fe4-4cb1-bbb6-74f396f1d835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:04.357393 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71535bff-c2d3-4830-9dc0-dd41cf434d94 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:04.357976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ad77b-2fe4-4cb1-bbb6-74f396f1d835 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:04.358257 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ad77b-2fe4-4cb1-bbb6-74f396f1d835 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:04.358575 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ad77b-2fe4-4cb1-bbb6-74f396f1d835 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:04.371893 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ad77b-2fe4-4cb1-bbb6-74f396f1d835 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:45:04.376931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-273ad77b-2fe4-4cb1-bbb6-74f396f1d835 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:45:04.376931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:04.376931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:04.376931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:04.376931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:04.376931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:04.376931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:04.376931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:04.376931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:04.376931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:04.376931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:04.376931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:04.376931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:45:04.376931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:45:04.376931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:45:04.376931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:45:04.377916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:45:04.377916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:45:04.377916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:45:04.377916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:45:04.377916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:04.377916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:04.377916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:04.377916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:04.377916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:04.377916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:04.377916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:04.377916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:04.377916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:04.377916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:04.377916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:04.377916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:04.378779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:04.378779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:04.378779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:04.378779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:04.378779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:45:04.378779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:45:04.378779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:45:04.378779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:45:04.378779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:45:04.378779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:45:04.378779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:45:04.378779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:04.378779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 321/642] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 31 10:45:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:04.379476 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ad77b-2fe4-4cb1-bbb6-74f396f1d835 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:04.480215 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71535bff-c2d3-4830-9dc0-dd41cf434d94 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:04.486188 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:04.496457 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ad77b-2fe4-4cb1-bbb6-74f396f1d835 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:04.507814 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:04.519980 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d37a84-5664-489e-8379-7b03f9dee21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:04.520614 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d37a84-5664-489e-8379-7b03f9dee21c None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:04.520890 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d37a84-5664-489e-8379-7b03f9dee21c None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:04.521188 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d37a84-5664-489e-8379-7b03f9dee21c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:04.530021 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d37a84-5664-489e-8379-7b03f9dee21c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:04.531491 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d37a84-5664-489e-8379-7b03f9dee21c None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:04.535118 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26278351-27ef-4518-af6d-74c2aebd2648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:04.535747 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26278351-27ef-4518-af6d-74c2aebd2648 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:04.536027 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26278351-27ef-4518-af6d-74c2aebd2648 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:04.536299 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26278351-27ef-4518-af6d-74c2aebd2648 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:04.543107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-e5d37a84-5664-489e-8379-7b03f9dee21c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 10:45:04.543107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:04.543107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:04.543107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:04.543107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:04.543107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:04.543107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:04.543107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:04.543107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:04.543107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:04.543107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:04.543107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:04.543107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:45:04.543107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:45:04.543107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:45:04.543107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:45:04.550271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:04.550271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:04.550271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:45:04.550271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:04.550271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:45:04.550271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:04.550271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:04.550271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:04.550271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:04.550271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:04.550271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:04.550271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:04.550271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:04.550271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:04.550271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:04.550271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:04.550271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:04.550271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:04.551272 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:04.551272 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:04.551272 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:45:04.551272 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:45:04.551272 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:45:04.551272 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:45:04.551272 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:45:04.551272 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:45:04.551272 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 10:45:04.551272 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:04.551272 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 322/643] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:45:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:04.551272 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d37a84-5664-489e-8379-7b03f9dee21c None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:04.557171 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26278351-27ef-4518-af6d-74c2aebd2648 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:04.559127 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26278351-27ef-4518-af6d-74c2aebd2648 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:04.575494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-26278351-27ef-4518-af6d-74c2aebd2648 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:04.575494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:04.575494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:04.575494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:04.575494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:04.575494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:04.575494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:04.575494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:04.575494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:04.575494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:04.575494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:04.575494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:04.575494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:04.575494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:04.575494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:04.575494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:04.576458 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:04.576458 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:04.576458 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:04.576458 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:04.576458 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:04.576458 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:04.576458 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:04.576458 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:04.576458 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:04.576458 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:04.576458 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:04.576458 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:04.576458 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:04.576458 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:04.576458 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:04.576458 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:04.576458 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:04.577499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:04.577499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:04.577499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:04.577499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 322/644] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:04.577499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26278351-27ef-4518-af6d-74c2aebd2648 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:04.665903 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d37a84-5664-489e-8379-7b03f9dee21c None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:04.691552 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26278351-27ef-4518-af6d-74c2aebd2648 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:04.705193 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47313897-510d-456d-ba48-519a6bf6fa19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:04.705862 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47313897-510d-456d-ba48-519a6bf6fa19 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:04.706095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47313897-510d-456d-ba48-519a6bf6fa19 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:04.706360 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47313897-510d-456d-ba48-519a6bf6fa19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:04.718240 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47313897-510d-456d-ba48-519a6bf6fa19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:04.719715 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47313897-510d-456d-ba48-519a6bf6fa19 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:04.730630 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636178b7-6f67-4408-b752-642876ff25f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:04.731499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636178b7-6f67-4408-b752-642876ff25f9 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:04.731881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636178b7-6f67-4408-b752-642876ff25f9 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:04.732252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636178b7-6f67-4408-b752-642876ff25f9 None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:04.732987 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636178b7-6f67-4408-b752-642876ff25f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:04.733289 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636178b7-6f67-4408-b752-642876ff25f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:04.734703 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636178b7-6f67-4408-b752-642876ff25f9 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:04.734924 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-47313897-510d-456d-ba48-519a6bf6fa19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:04.734924 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:04.734924 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:04.734924 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:04.734924 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:04.734924 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:04.734924 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:04.734924 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:04.734924 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:04.734924 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:04.734924 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:04.734924 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:04.734924 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:04.734924 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:04.734924 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:04.734924 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:04.735805 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:04.735805 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:04.735805 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:04.735805 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:04.735805 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:04.735805 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:04.735805 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:04.735805 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:04.735805 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:04.735805 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:04.735805 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:04.735805 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:04.735805 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:04.735805 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:04.735805 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:04.735805 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:04.735805 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:04.736676 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:04.736676 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:04.736676 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:04.736676 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 323/645] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:04.737474 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47313897-510d-456d-ba48-519a6bf6fa19 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:04.748536 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 323/646] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 31 10:45:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:04.749232 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636178b7-6f67-4408-b752-642876ff25f9 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:04.759127 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:04.780639 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:04.852925 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47313897-510d-456d-ba48-519a6bf6fa19 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:04.860424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636178b7-6f67-4408-b752-642876ff25f9 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:04.889538 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4654fff-4588-4cc0-b0a4-75fd3c9d9887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:04.890200 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4654fff-4588-4cc0-b0a4-75fd3c9d9887 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:04.890447 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4654fff-4588-4cc0-b0a4-75fd3c9d9887 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:04.890720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4654fff-4588-4cc0-b0a4-75fd3c9d9887 None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:04.891455 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4654fff-4588-4cc0-b0a4-75fd3c9d9887 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:04.891763 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4654fff-4588-4cc0-b0a4-75fd3c9d9887 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:04.893292 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4654fff-4588-4cc0-b0a4-75fd3c9d9887 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:04.898112 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69775497-c918-413c-83ae-0101f93a5195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:04.898758 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69775497-c918-413c-83ae-0101f93a5195 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:04.899034 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69775497-c918-413c-83ae-0101f93a5195 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:04.899303 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69775497-c918-413c-83ae-0101f93a5195 None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:04.899950 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69775497-c918-413c-83ae-0101f93a5195 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:04.900249 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69775497-c918-413c-83ae-0101f93a5195 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:04.901717 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69775497-c918-413c-83ae-0101f93a5195 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:04.917786 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 324/647] 10.222.0.16 () {56 vars in 1137 bytes} [Wed Aug 31 10:45:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:04.917786 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4654fff-4588-4cc0-b0a4-75fd3c9d9887 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:04.921076 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 324/648] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:04.925661 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69775497-c918-413c-83ae-0101f93a5195 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:04.998606 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:05.019237 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:05.027729 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4654fff-4588-4cc0-b0a4-75fd3c9d9887 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:05.035515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69775497-c918-413c-83ae-0101f93a5195 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:05.067853 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c332be-1153-4db4-9053-1ecb40498dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:05.068454 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c332be-1153-4db4-9053-1ecb40498dc6 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:05.068727 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c332be-1153-4db4-9053-1ecb40498dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:05.069000 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c332be-1153-4db4-9053-1ecb40498dc6 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:05.069634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c332be-1153-4db4-9053-1ecb40498dc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:05.069931 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c332be-1153-4db4-9053-1ecb40498dc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:05.071312 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c332be-1153-4db4-9053-1ecb40498dc6 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:05.072928 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfafaf2-a620-4df3-aa9c-023aa7745eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:05.073774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfafaf2-a620-4df3-aa9c-023aa7745eeb None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:05.074052 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfafaf2-a620-4df3-aa9c-023aa7745eeb None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:05.074329 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfafaf2-a620-4df3-aa9c-023aa7745eeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:05.083727 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 325/649] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:05.083727 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c332be-1153-4db4-9053-1ecb40498dc6 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:05.091708 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfafaf2-a620-4df3-aa9c-023aa7745eeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:05.093170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfafaf2-a620-4df3-aa9c-023aa7745eeb None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:05.116760 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-1bfafaf2-a620-4df3-aa9c-023aa7745eeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:05.116760 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:05.116760 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:05.116760 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:05.116760 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:05.116760 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:05.116760 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:05.116760 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:05.116760 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:05.116760 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:05.116760 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:05.116760 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:05.116760 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:05.116760 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:05.116760 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:05.116760 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:05.117721 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:05.117721 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:05.117721 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:05.117721 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:05.117721 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:05.117721 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:05.117721 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:05.117721 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:05.117721 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:05.117721 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:05.117721 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:05.117721 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:05.117721 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:05.117721 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:05.117721 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:05.117721 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:05.117721 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:05.118682 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:05.118682 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:05.118682 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:05.118682 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 325/650] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:05.118923 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfafaf2-a620-4df3-aa9c-023aa7745eeb None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:05.208446 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c332be-1153-4db4-9053-1ecb40498dc6 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:05.227059 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfafaf2-a620-4df3-aa9c-023aa7745eeb None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:05.244263 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d75912c-ae07-41eb-9f9c-b28dc76a7062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:05.245130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d75912c-ae07-41eb-9f9c-b28dc76a7062 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:05.245402 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d75912c-ae07-41eb-9f9c-b28dc76a7062 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:05.245672 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d75912c-ae07-41eb-9f9c-b28dc76a7062 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:05.245866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:05.263905 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5702e48-0e6e-4fa9-8d5b-6a40b1d87079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:05.264528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5702e48-0e6e-4fa9-8d5b-6a40b1d87079 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:05.264803 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5702e48-0e6e-4fa9-8d5b-6a40b1d87079 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:05.265072 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5702e48-0e6e-4fa9-8d5b-6a40b1d87079 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:05.268111 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:05.271870 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5702e48-0e6e-4fa9-8d5b-6a40b1d87079 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:05.273340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5702e48-0e6e-4fa9-8d5b-6a40b1d87079 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:05.285212 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-f5702e48-0e6e-4fa9-8d5b-6a40b1d87079 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:05.285212 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:05.285212 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:05.285212 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:05.285212 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:05.285212 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:05.285212 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:05.285212 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:05.285212 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:05.285212 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:05.285212 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:05.285212 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:05.285212 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:45:05.285212 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:45:05.285212 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:45:05.286280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:45:05.286280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:05.286280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:05.286280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:05.286280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:05.286280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:05.286280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:05.286280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:05.286280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:05.286280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:05.286280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:05.286280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:05.286280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:05.286280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:05.286280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:05.286280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:05.286280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:05.287659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:45:05.287659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:45:05.287659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:45:05.287659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:45:05.287659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:45:05.287659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:45:05.287659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:05.287659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:05.287659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 326/651] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:45:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:05.287659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5702e48-0e6e-4fa9-8d5b-6a40b1d87079 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:05.314935 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d75912c-ae07-41eb-9f9c-b28dc76a7062 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:05.316560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d75912c-ae07-41eb-9f9c-b28dc76a7062 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:05.394619 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5702e48-0e6e-4fa9-8d5b-6a40b1d87079 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:05.419567 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 326/652] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:45:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:45:05.420334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d75912c-ae07-41eb-9f9c-b28dc76a7062 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:05.432822 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd96441e-dad3-4ee8-a18b-6bc18827f4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:05.433459 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96441e-dad3-4ee8-a18b-6bc18827f4d5 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:05.433741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96441e-dad3-4ee8-a18b-6bc18827f4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:05.434668 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96441e-dad3-4ee8-a18b-6bc18827f4d5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:05.442792 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd96441e-dad3-4ee8-a18b-6bc18827f4d5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:05.444238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd96441e-dad3-4ee8-a18b-6bc18827f4d5 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:05.455686 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-cd96441e-dad3-4ee8-a18b-6bc18827f4d5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 10:45:05.455686 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:05.455686 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:05.455686 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:05.455686 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:05.455686 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:05.455686 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:05.455686 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:05.455686 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:05.455686 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:05.455686 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:05.455686 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:05.455686 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:45:05.455686 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:45:05.455686 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:45:05.455686 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:45:05.456845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:05.456845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:05.456845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:45:05.456845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:05.456845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:45:05.456845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:05.456845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:05.456845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:05.456845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:05.456845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:05.456845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:05.456845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:05.456845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:05.456845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:05.456845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:05.456845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:05.456845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:05.456845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:05.457914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:05.457914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:05.457914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:45:05.457914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:45:05.457914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:45:05.457914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:45:05.457914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:45:05.457914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:45:05.457914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 10:45:05.457914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:05.457914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 327/653] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:45:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:05.458657 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9147a8-1c43-41e8-aee2-38baa42c1ddf None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:05.458657 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9147a8-1c43-41e8-aee2-38baa42c1ddf None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:05.458836 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9147a8-1c43-41e8-aee2-38baa42c1ddf None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:05.460072 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 328/654] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:05.460760 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9147a8-1c43-41e8-aee2-38baa42c1ddf None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:05.479293 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:05.505821 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:05.528929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d75912c-ae07-41eb-9f9c-b28dc76a7062 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:05.563789 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d9e90-19b4-4969-9a8c-c801df9b3f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:05.564398 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d9e90-19b4-4969-9a8c-c801df9b3f77 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:05.564670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d9e90-19b4-4969-9a8c-c801df9b3f77 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:05.564937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d9e90-19b4-4969-9a8c-c801df9b3f77 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:05.565574 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d9e90-19b4-4969-9a8c-c801df9b3f77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:05.565873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d9e90-19b4-4969-9a8c-c801df9b3f77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:05.566513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9147a8-1c43-41e8-aee2-38baa42c1ddf None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:05.567274 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d9e90-19b4-4969-9a8c-c801df9b3f77 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:05.579734 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 327/655] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:05.580550 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d9e90-19b4-4969-9a8c-c801df9b3f77 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:05.603259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44931202-9cfe-4be5-9765-06296ce03b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:05.603871 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44931202-9cfe-4be5-9765-06296ce03b1e None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:05.604143 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44931202-9cfe-4be5-9765-06296ce03b1e None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:05.604413 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44931202-9cfe-4be5-9765-06296ce03b1e None admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:05.605105 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44931202-9cfe-4be5-9765-06296ce03b1e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:05.605411 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44931202-9cfe-4be5-9765-06296ce03b1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:05.607896 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44931202-9cfe-4be5-9765-06296ce03b1e None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:05.619727 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 329/656] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:45:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:05.620740 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44931202-9cfe-4be5-9765-06296ce03b1e None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:05.683630 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d9e90-19b4-4969-9a8c-c801df9b3f77 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:05.697649 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:05.718845 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d7196-9154-4561-b4d6-976c3252563e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:05.719385 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:05.719728 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d7196-9154-4561-b4d6-976c3252563e None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:05.722646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d7196-9154-4561-b4d6-976c3252563e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:05.722646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d7196-9154-4561-b4d6-976c3252563e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:05.723334 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44931202-9cfe-4be5-9765-06296ce03b1e None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:05.730587 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d7196-9154-4561-b4d6-976c3252563e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:05.732791 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d7196-9154-4561-b4d6-976c3252563e None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:05.749722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-c79d7196-9154-4561-b4d6-976c3252563e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:05.749722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:05.749722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:05.749722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:05.749722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:05.749722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:05.749722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:05.749722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:05.749722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:05.749722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:05.749722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:05.749722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:05.749722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:05.749722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:05.749722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:05.749722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:05.750955 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:05.750955 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:05.750955 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:05.750955 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:05.750955 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:05.750955 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:05.750955 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:05.750955 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:05.750955 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:05.750955 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:05.750955 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:05.750955 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:05.750955 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:05.750955 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:05.750955 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:05.750955 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:05.750955 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:05.751873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:05.751873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:05.751873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:05.751873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 328/657] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:05.752367 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff85aa-1538-47fa-b9de-1de8a2fd7b87 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:05.752645 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff85aa-1538-47fa-b9de-1de8a2fd7b87 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:05.752917 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff85aa-1538-47fa-b9de-1de8a2fd7b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:05.761117 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990af62-6157-47f8-a89d-572338bed693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:05.761787 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990af62-6157-47f8-a89d-572338bed693 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:05.762208 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990af62-6157-47f8-a89d-572338bed693 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:05.769016 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990af62-6157-47f8-a89d-572338bed693 None admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:05.769016 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2990af62-6157-47f8-a89d-572338bed693 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:05.769016 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2990af62-6157-47f8-a89d-572338bed693 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:05.769016 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2990af62-6157-47f8-a89d-572338bed693 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:05.771763 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-d9ff85aa-1538-47fa-b9de-1de8a2fd7b87 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:05.772982 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-d9ff85aa-1538-47fa-b9de-1de8a2fd7b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:05.783133 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 330/658] 10.222.0.16 () {56 vars in 1137 bytes} [Wed Aug 31 10:45:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:05.784263 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990af62-6157-47f8-a89d-572338bed693 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:05.845453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 329/659] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:45:05.845453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff85aa-1538-47fa-b9de-1de8a2fd7b87 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:05.898977 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990af62-6157-47f8-a89d-572338bed693 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:05.938089 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1eeaeb-cd5c-4390-884a-ef84f298d13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:05.938694 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1eeaeb-cd5c-4390-884a-ef84f298d13d None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:05.939004 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1eeaeb-cd5c-4390-884a-ef84f298d13d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:05.939276 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1eeaeb-cd5c-4390-884a-ef84f298d13d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:05.946994 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1eeaeb-cd5c-4390-884a-ef84f298d13d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:05.948446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1eeaeb-cd5c-4390-884a-ef84f298d13d None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:05.950169 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:05.957015 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff85aa-1538-47fa-b9de-1de8a2fd7b87 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:05.961230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-9b1eeaeb-cd5c-4390-884a-ef84f298d13d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 10:45:05.961230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:05.961230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:05.961230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:05.961230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:05.961230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:05.961230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:05.961230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:05.961230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:05.961230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:05.961230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:05.961230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:05.961230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:45:05.961230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:45:05.961230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:45:05.961230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:45:05.964106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:05.964106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:05.964106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:45:05.964106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:05.964106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:45:05.964106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:05.964106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:05.964106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:05.964106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:05.964106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:05.964106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:05.964106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:05.964106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:05.964106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:05.964106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:05.964106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:05.964106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:05.964106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:05.965303 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:05.965303 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:05.965303 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:45:05.965303 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:45:05.965303 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:45:05.965303 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:45:05.965303 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:45:05.965303 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:45:05.965303 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 10:45:05.965303 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:05.965303 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 331/660] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 31 10:45:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:05.965303 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1eeaeb-cd5c-4390-884a-ef84f298d13d None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:05.991182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:06.007919 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fbaf0e-c889-48c3-b68d-32c922a06c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:06.009576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbaf0e-c889-48c3-b68d-32c922a06c08 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:06.010006 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbaf0e-c889-48c3-b68d-32c922a06c08 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:06.010462 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbaf0e-c889-48c3-b68d-32c922a06c08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:06.011365 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fbaf0e-c889-48c3-b68d-32c922a06c08 None admin] RBAC: Adding query filter params (role.id=bec755d6565a41798b09f5f57c1d51dd, scope.project.id=8db9b3f8a9f041938ac7b39003a35015, user.id=f11971a594014d86968c3356d11d42f7) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:06.011818 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fbaf0e-c889-48c3-b68d-32c922a06c08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:06.015037 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fbaf0e-c889-48c3-b68d-32c922a06c08 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:06.039601 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 330/661] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:45:05 2022] GET /identity/v3/role_assignments?user.id=f11971a594014d86968c3356d11d42f7&scope.project.id=8db9b3f8a9f041938ac7b39003a35015&role.id=bec755d6565a41798b09f5f57c1d51dd => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:06.039601 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fbaf0e-c889-48c3-b68d-32c922a06c08 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:06.102952 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1eeaeb-cd5c-4390-884a-ef84f298d13d None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:06.145601 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a8b8b-4d87-4fb7-9bba-328bd464ddcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:06.146225 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a8b8b-4d87-4fb7-9bba-328bd464ddcf None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:06.146543 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a8b8b-4d87-4fb7-9bba-328bd464ddcf None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:06.146813 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a8b8b-4d87-4fb7-9bba-328bd464ddcf None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:06.147466 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145a8b8b-4d87-4fb7-9bba-328bd464ddcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:06.147766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145a8b8b-4d87-4fb7-9bba-328bd464ddcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:06.149164 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145a8b8b-4d87-4fb7-9bba-328bd464ddcf None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:06.162771 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 332/662] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:06.163071 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a8b8b-4d87-4fb7-9bba-328bd464ddcf None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:06.183625 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fbaf0e-c889-48c3-b68d-32c922a06c08 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:06.237805 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3657d4d3-777a-46c4-aa03-7c32e44df6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:06.238353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657d4d3-777a-46c4-aa03-7c32e44df6bf None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:06.238705 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657d4d3-777a-46c4-aa03-7c32e44df6bf None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:06.238975 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657d4d3-777a-46c4-aa03-7c32e44df6bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:06.266275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:06.271608 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:06.290545 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a8b8b-4d87-4fb7-9bba-328bd464ddcf None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:06.321923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3657d4d3-777a-46c4-aa03-7c32e44df6bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:06.323453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3657d4d3-777a-46c4-aa03-7c32e44df6bf None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:06.331750 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809be40c-ddf2-4d4b-8207-a3b9f803a841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:06.332350 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809be40c-ddf2-4d4b-8207-a3b9f803a841 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:06.332631 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809be40c-ddf2-4d4b-8207-a3b9f803a841 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:06.332905 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809be40c-ddf2-4d4b-8207-a3b9f803a841 None admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:06.333670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809be40c-ddf2-4d4b-8207-a3b9f803a841 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:06.334075 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809be40c-ddf2-4d4b-8207-a3b9f803a841 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:06.336178 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809be40c-ddf2-4d4b-8207-a3b9f803a841 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:06.360269 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 333/663] 10.222.0.16 () {56 vars in 1139 bytes} [Wed Aug 31 10:45:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:06.360269 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809be40c-ddf2-4d4b-8207-a3b9f803a841 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:06.440932 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 331/664] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:45:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:45:06.441749 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3657d4d3-777a-46c4-aa03-7c32e44df6bf None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:06.463218 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809be40c-ddf2-4d4b-8207-a3b9f803a841 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:06.500323 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc7521-0dd4-42d5-8d31-dbf99b5481fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:06.500882 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc7521-0dd4-42d5-8d31-dbf99b5481fe None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:06.501161 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc7521-0dd4-42d5-8d31-dbf99b5481fe None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:06.506441 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc7521-0dd4-42d5-8d31-dbf99b5481fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:06.506441 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:06.513426 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc7521-0dd4-42d5-8d31-dbf99b5481fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:45:06.515769 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:06.518528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-97cc7521-0dd4-42d5-8d31-dbf99b5481fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:45:06.518528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:06.518528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:06.518528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:06.518528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:06.518528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:06.518528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:06.518528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:06.518528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:06.518528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:06.518528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:06.518528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:06.518528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:45:06.518528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:45:06.518528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:45:06.518528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:45:06.519659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:45:06.519659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:45:06.519659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:45:06.519659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:45:06.519659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:06.519659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:06.519659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:06.519659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:06.519659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:06.519659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:06.519659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:06.519659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:06.519659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:06.519659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:06.519659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:06.519659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:06.520938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:06.520938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:06.520938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:06.520938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:06.520938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:45:06.520938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:45:06.520938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:45:06.520938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:45:06.520938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:45:06.520938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:45:06.520938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:45:06.520938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:06.524671 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 334/665] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 31 10:45:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:06.525573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc7521-0dd4-42d5-8d31-dbf99b5481fe None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:06.564270 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3657d4d3-777a-46c4-aa03-7c32e44df6bf None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:06.603548 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abbd700-a6cc-40d4-ab94-e79a5d9d9a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:06.604218 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abbd700-a6cc-40d4-ab94-e79a5d9d9a5a None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:06.604496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abbd700-a6cc-40d4-ab94-e79a5d9d9a5a None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:06.604773 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abbd700-a6cc-40d4-ab94-e79a5d9d9a5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:06.612834 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abbd700-a6cc-40d4-ab94-e79a5d9d9a5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:06.614310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abbd700-a6cc-40d4-ab94-e79a5d9d9a5a None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:06.627228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-9abbd700-a6cc-40d4-ab94-e79a5d9d9a5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:06.627228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:06.627228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:06.627228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:06.627228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:06.627228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:06.627228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:06.627228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:06.627228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:06.627228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:06.627228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:06.627228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:06.627228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:45:06.627228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:45:06.627228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:45:06.628172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:45:06.628172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:06.628172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:06.628172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:06.628172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:06.628172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:06.628172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:06.628172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:06.628172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:06.628172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:06.628172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:06.628172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:06.628172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:06.628172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:06.628172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:06.628172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:06.628172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:06.629067 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:45:06.629067 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:45:06.629067 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:45:06.629067 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:45:06.629067 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:45:06.629067 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:45:06.629067 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:06.629067 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:06.633210 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 332/666] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:45:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:06.634082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abbd700-a6cc-40d4-ab94-e79a5d9d9a5a None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:06.645666 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc7521-0dd4-42d5-8d31-dbf99b5481fe None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:06.684854 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf975610-691d-43df-ab5a-047e183c45ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:06.685461 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf975610-691d-43df-ab5a-047e183c45ed None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:06.685774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf975610-691d-43df-ab5a-047e183c45ed None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:06.686073 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf975610-691d-43df-ab5a-047e183c45ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:06.740745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abbd700-a6cc-40d4-ab94-e79a5d9d9a5a None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:06.753864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:06.759324 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:06.761461 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf975610-691d-43df-ab5a-047e183c45ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:06.763014 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf975610-691d-43df-ab5a-047e183c45ed None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:06.776758 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379ac04-c388-41fe-89eb-29babdf2f086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:06.777389 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379ac04-c388-41fe-89eb-29babdf2f086 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:06.778190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379ac04-c388-41fe-89eb-29babdf2f086 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:06.778582 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379ac04-c388-41fe-89eb-29babdf2f086 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:06.789381 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3379ac04-c388-41fe-89eb-29babdf2f086 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:06.790869 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3379ac04-c388-41fe-89eb-29babdf2f086 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:06.806165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-3379ac04-c388-41fe-89eb-29babdf2f086 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:06.806165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:06.806165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:06.806165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:06.806165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:06.806165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:06.806165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:06.806165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:06.806165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:06.806165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:06.806165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:06.806165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:06.806165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:06.806165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:06.806165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:06.806165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:06.807182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:06.807182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:06.807182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:06.807182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:06.807182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:06.807182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:06.807182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:06.807182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:06.807182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:06.807182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:06.807182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:06.807182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:06.807182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:06.807182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:06.807182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:06.807182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:06.807182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:06.808039 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:06.808039 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:06.808039 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:06.808039 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 333/667] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:06.808278 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379ac04-c388-41fe-89eb-29babdf2f086 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:06.879973 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 335/668] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:45:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:45:06.881182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf975610-691d-43df-ab5a-047e183c45ed None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:06.933387 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379ac04-c388-41fe-89eb-29babdf2f086 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:06.970187 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb4ed6-60fa-4e8b-90fd-b7405cb3d2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:06.970831 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb4ed6-60fa-4e8b-90fd-b7405cb3d2b8 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:06.971111 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb4ed6-60fa-4e8b-90fd-b7405cb3d2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:06.971569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb4ed6-60fa-4e8b-90fd-b7405cb3d2b8 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:06.972045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb4ed6-60fa-4e8b-90fd-b7405cb3d2b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:06.972349 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb4ed6-60fa-4e8b-90fd-b7405cb3d2b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:06.973992 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb4ed6-60fa-4e8b-90fd-b7405cb3d2b8 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:06.983705 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 334/669] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 31 10:45:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:06.984528 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb4ed6-60fa-4e8b-90fd-b7405cb3d2b8 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:06.994053 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:07.002096 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf975610-691d-43df-ab5a-047e183c45ed None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:07.019520 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:07.038980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf32a9-aa49-4d0e-aef6-c73e4ff5550d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:07.039734 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf32a9-aa49-4d0e-aef6-c73e4ff5550d None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:07.040056 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf32a9-aa49-4d0e-aef6-c73e4ff5550d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:07.040428 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf32a9-aa49-4d0e-aef6-c73e4ff5550d None admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:07.041214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cf32a9-aa49-4d0e-aef6-c73e4ff5550d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:07.041632 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cf32a9-aa49-4d0e-aef6-c73e4ff5550d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:07.043745 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cf32a9-aa49-4d0e-aef6-c73e4ff5550d None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:07.062054 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 336/670] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:45:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:07.062054 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf32a9-aa49-4d0e-aef6-c73e4ff5550d None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:07.103067 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb4ed6-60fa-4e8b-90fd-b7405cb3d2b8 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:07.150093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca1d1c-abe9-46d7-ab52-b7b68a6ea517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:07.150727 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca1d1c-abe9-46d7-ab52-b7b68a6ea517 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:07.151001 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca1d1c-abe9-46d7-ab52-b7b68a6ea517 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:07.151265 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca1d1c-abe9-46d7-ab52-b7b68a6ea517 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:07.151905 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eca1d1c-abe9-46d7-ab52-b7b68a6ea517 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:07.152198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eca1d1c-abe9-46d7-ab52-b7b68a6ea517 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:07.153816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eca1d1c-abe9-46d7-ab52-b7b68a6ea517 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:07.171872 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 335/671] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:07.171872 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca1d1c-abe9-46d7-ab52-b7b68a6ea517 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:07.175022 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf32a9-aa49-4d0e-aef6-c73e4ff5550d None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:07.209279 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7b369-d1b8-413c-8e5c-e28ec13d3dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:07.209910 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7b369-d1b8-413c-8e5c-e28ec13d3dd8 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:07.210188 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7b369-d1b8-413c-8e5c-e28ec13d3dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:07.210556 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7b369-d1b8-413c-8e5c-e28ec13d3dd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:07.222429 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c7b369-d1b8-413c-8e5c-e28ec13d3dd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:07.223910 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c7b369-d1b8-413c-8e5c-e28ec13d3dd8 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:07.239786 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-89c7b369-d1b8-413c-8e5c-e28ec13d3dd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:07.239786 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:07.239786 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:07.239786 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:07.239786 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:07.239786 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:07.239786 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:07.239786 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:07.239786 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:07.239786 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:07.239786 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:07.239786 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:07.239786 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:07.239786 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:07.239786 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:07.239786 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:07.241007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:07.241007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:07.241007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:07.241007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:07.241007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:07.241007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:07.241007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:07.241007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:07.241007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:07.241007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:07.241007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:07.241007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:07.241007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:07.241007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:07.241007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:07.241007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:07.241007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:07.242156 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:07.242156 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:07.242156 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:07.245805 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 337/672] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:07.246665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7b369-d1b8-413c-8e5c-e28ec13d3dd8 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:07.256431 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:07.273032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:07.283649 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca1d1c-abe9-46d7-ab52-b7b68a6ea517 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:07.322750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad4ea6-69d5-4567-a7f5-6ab5681b545b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:07.323373 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad4ea6-69d5-4567-a7f5-6ab5681b545b None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:07.323646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad4ea6-69d5-4567-a7f5-6ab5681b545b None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:07.323915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad4ea6-69d5-4567-a7f5-6ab5681b545b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:07.335130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad4ea6-69d5-4567-a7f5-6ab5681b545b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:07.339085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad4ea6-69d5-4567-a7f5-6ab5681b545b None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:07.351993 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7b369-d1b8-413c-8e5c-e28ec13d3dd8 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:07.353622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-6cad4ea6-69d5-4567-a7f5-6ab5681b545b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:07.353622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:07.353622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:07.353622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:07.353622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:07.353622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:07.353622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:07.353622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:07.353622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:07.353622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:07.353622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:07.353622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:07.353622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:07.353622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:07.353622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:07.353622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:07.354754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:07.354754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:07.354754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:07.354754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:07.354754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:07.354754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:07.354754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:07.354754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:07.354754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:07.354754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:07.354754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:07.354754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:07.354754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:07.354754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:07.354754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:07.354754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:07.354754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:07.356000 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:07.356000 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:07.356000 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:07.360004 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 336/673] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:07.360721 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad4ea6-69d5-4567-a7f5-6ab5681b545b None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:07.393185 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b45da1-9200-4385-a713-16ef4dc67ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:07.393884 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b45da1-9200-4385-a713-16ef4dc67ffd None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:07.394227 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b45da1-9200-4385-a713-16ef4dc67ffd None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:07.394606 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b45da1-9200-4385-a713-16ef4dc67ffd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:07.395407 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b45da1-9200-4385-a713-16ef4dc67ffd None admin] RBAC: Adding query filter params (role.id=bec755d6565a41798b09f5f57c1d51dd, scope.project.id=8db9b3f8a9f041938ac7b39003a35015, user.id=49f9aeb9857b45d68c3baa8fc4acc5fb) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:07.395790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b45da1-9200-4385-a713-16ef4dc67ffd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:07.397285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b45da1-9200-4385-a713-16ef4dc67ffd None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:07.403593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 338/674] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:45:07 2022] GET /identity/v3/role_assignments?user.id=49f9aeb9857b45d68c3baa8fc4acc5fb&scope.project.id=8db9b3f8a9f041938ac7b39003a35015&role.id=bec755d6565a41798b09f5f57c1d51dd => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:07.404478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b45da1-9200-4385-a713-16ef4dc67ffd None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:07.500501 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad4ea6-69d5-4567-a7f5-6ab5681b545b None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:07.512544 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:07.542797 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97635094-3e2d-4cb6-abe0-70c13f207a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:07.543889 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97635094-3e2d-4cb6-abe0-70c13f207a26 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:07.544424 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97635094-3e2d-4cb6-abe0-70c13f207a26 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:07.544858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97635094-3e2d-4cb6-abe0-70c13f207a26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:07.558372 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97635094-3e2d-4cb6-abe0-70c13f207a26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:07.561438 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97635094-3e2d-4cb6-abe0-70c13f207a26 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:07.564855 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b45da1-9200-4385-a713-16ef4dc67ffd None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:07.578091 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-97635094-3e2d-4cb6-abe0-70c13f207a26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:07.578091 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:07.578091 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:07.578091 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:07.578091 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:07.578091 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:07.578091 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:07.578091 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:07.578091 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:07.578091 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:07.578091 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:07.578091 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:07.578091 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:45:07.578091 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:45:07.578091 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:45:07.579068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:45:07.579068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:07.579068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:07.579068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:07.579068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:07.579068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:07.579068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:07.579068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:07.579068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:07.579068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:07.579068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:07.579068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:07.579068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:07.579068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:07.579068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:07.579068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:07.579068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:07.580143 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:45:07.580143 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:45:07.580143 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:45:07.580143 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:45:07.580143 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:45:07.580143 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:45:07.580143 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:07.580143 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:07.586465 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 337/675] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:45:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:07.587470 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97635094-3e2d-4cb6-abe0-70c13f207a26 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:07.598148 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:07.611669 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173790ba-4aaf-478b-b700-316dc565522b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:07.612268 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173790ba-4aaf-478b-b700-316dc565522b None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:07.612547 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173790ba-4aaf-478b-b700-316dc565522b None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:07.613243 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173790ba-4aaf-478b-b700-316dc565522b None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:07.614718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173790ba-4aaf-478b-b700-316dc565522b None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:07.614718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173790ba-4aaf-478b-b700-316dc565522b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:07.615258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173790ba-4aaf-478b-b700-316dc565522b None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:07.635809 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 339/676] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:07.635809 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173790ba-4aaf-478b-b700-316dc565522b None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:07.740090 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97635094-3e2d-4cb6-abe0-70c13f207a26 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:07.754471 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173790ba-4aaf-478b-b700-316dc565522b None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:07.787139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:07.790734 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8b42b-c49c-4808-a58e-c7b46fc1d3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:07.791431 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8b42b-c49c-4808-a58e-c7b46fc1d3b4 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:07.791773 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8b42b-c49c-4808-a58e-c7b46fc1d3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:07.792108 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8b42b-c49c-4808-a58e-c7b46fc1d3b4 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:07.792874 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df8b42b-c49c-4808-a58e-c7b46fc1d3b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:07.793267 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df8b42b-c49c-4808-a58e-c7b46fc1d3b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:07.794646 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1125d240-2e32-432a-ac5c-48b51069ea27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:07.794798 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df8b42b-c49c-4808-a58e-c7b46fc1d3b4 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:07.795661 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125d240-2e32-432a-ac5c-48b51069ea27 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:07.796089 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125d240-2e32-432a-ac5c-48b51069ea27 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:07.796504 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125d240-2e32-432a-ac5c-48b51069ea27 None admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:07.797348 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1125d240-2e32-432a-ac5c-48b51069ea27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:07.797800 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1125d240-2e32-432a-ac5c-48b51069ea27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:07.799578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1125d240-2e32-432a-ac5c-48b51069ea27 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:07.812745 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 340/678] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:45:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:07.813830 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1125d240-2e32-432a-ac5c-48b51069ea27 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:07.816823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 338/678] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:07.822451 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8b42b-c49c-4808-a58e-c7b46fc1d3b4 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:07.896378 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:07.941917 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8b42b-c49c-4808-a58e-c7b46fc1d3b4 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:07.960507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1125d240-2e32-432a-ac5c-48b51069ea27 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:07.981425 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a7538-cb7d-428a-8f94-a813ad3ecfc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:07.982415 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a7538-cb7d-428a-8f94-a813ad3ecfc1 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:07.982866 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a7538-cb7d-428a-8f94-a813ad3ecfc1 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:07.983305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a7538-cb7d-428a-8f94-a813ad3ecfc1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:07.993228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a7538-cb7d-428a-8f94-a813ad3ecfc1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:07.995425 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a7538-cb7d-428a-8f94-a813ad3ecfc1 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:08.003023 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953acbb6-0f58-4e20-98d1-8be98e7db1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:08.003943 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953acbb6-0f58-4e20-98d1-8be98e7db1d2 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:08.004455 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953acbb6-0f58-4e20-98d1-8be98e7db1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:08.004896 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953acbb6-0f58-4e20-98d1-8be98e7db1d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:08.009942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-102a7538-cb7d-428a-8f94-a813ad3ecfc1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 10:45:08.009942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:08.009942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:08.009942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:08.009942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:08.009942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:08.009942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:08.009942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:08.009942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:08.009942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:08.009942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:08.009942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:08.009942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:45:08.009942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:45:08.009942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:45:08.009942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:45:08.010978 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:08.010978 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:08.010978 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:45:08.010978 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:08.010978 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:45:08.010978 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:08.010978 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:08.010978 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:08.010978 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:08.010978 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:08.010978 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:08.010978 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:08.010978 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:08.010978 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:08.010978 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:08.010978 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:08.010978 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:08.010978 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:08.012428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:08.012428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:08.012428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:45:08.012428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:45:08.012428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:45:08.012428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:45:08.012428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:45:08.012428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:45:08.012428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 10:45:08.012428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:08.012428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 339/679] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:45:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:08.017606 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953acbb6-0f58-4e20-98d1-8be98e7db1d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:08.017764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a7538-cb7d-428a-8f94-a813ad3ecfc1 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:08.020058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953acbb6-0f58-4e20-98d1-8be98e7db1d2 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:08.035041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-953acbb6-0f58-4e20-98d1-8be98e7db1d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:08.035041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:08.035041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:08.035041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:08.035041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:08.035041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:08.035041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:08.035041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:08.035041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:08.035041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:08.035041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:08.035041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:08.035041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:45:08.035041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:45:08.035041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:45:08.036194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:45:08.036194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:08.036194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:08.036194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:08.036194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:08.036194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:08.036194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:08.036194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:08.036194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:08.036194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:08.036194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:08.036194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:08.036194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:08.036194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:08.036194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:08.036194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:08.036194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:08.037149 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:45:08.037149 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:45:08.037149 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:45:08.037149 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:45:08.037149 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:45:08.037149 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:45:08.037149 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:08.037149 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:08.037149 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 341/680] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:45:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:08.037709 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953acbb6-0f58-4e20-98d1-8be98e7db1d2 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:08.098027 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:08.147835 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a7538-cb7d-428a-8f94-a813ad3ecfc1 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:08.157426 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953acbb6-0f58-4e20-98d1-8be98e7db1d2 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:08.174966 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:08.186276 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765294a-3d12-43ad-98bf-1d90ca7135f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:08.186903 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765294a-3d12-43ad-98bf-1d90ca7135f7 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:08.187173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765294a-3d12-43ad-98bf-1d90ca7135f7 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:08.187438 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765294a-3d12-43ad-98bf-1d90ca7135f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:08.188263 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765294a-3d12-43ad-98bf-1d90ca7135f7 None admin] RBAC: Adding query filter params (role.id=bec755d6565a41798b09f5f57c1d51dd, scope.project.id=8db9b3f8a9f041938ac7b39003a35015, user.id=f2e72c8e7b444e978bf99c6404febcd2) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:08.188564 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765294a-3d12-43ad-98bf-1d90ca7135f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:08.190109 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765294a-3d12-43ad-98bf-1d90ca7135f7 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:08.193564 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a89e169-f419-4067-ad1f-3810f0f619f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:08.194284 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a89e169-f419-4067-ad1f-3810f0f619f0 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:08.194684 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a89e169-f419-4067-ad1f-3810f0f619f0 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:08.195050 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a89e169-f419-4067-ad1f-3810f0f619f0 None admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:08.195877 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a89e169-f419-4067-ad1f-3810f0f619f0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:08.206519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a89e169-f419-4067-ad1f-3810f0f619f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:08.206737 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 340/681] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:45:08 2022] GET /identity/v3/role_assignments?user.id=f2e72c8e7b444e978bf99c6404febcd2&scope.project.id=8db9b3f8a9f041938ac7b39003a35015&role.id=bec755d6565a41798b09f5f57c1d51dd => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:08.206737 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765294a-3d12-43ad-98bf-1d90ca7135f7 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:08.206937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a89e169-f419-4067-ad1f-3810f0f619f0 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:08.235370 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 342/682] 10.222.0.16 () {56 vars in 1135 bytes} [Wed Aug 31 10:45:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:08.236962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a8f6a-c2e4-49c3-9ccf-713f3c99a1c3 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:08.237350 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a8f6a-c2e4-49c3-9ccf-713f3c99a1c3 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:08.237721 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a8f6a-c2e4-49c3-9ccf-713f3c99a1c3 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:08.242561 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 343/683] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:08.243560 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a8f6a-c2e4-49c3-9ccf-713f3c99a1c3 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:08.344119 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765294a-3d12-43ad-98bf-1d90ca7135f7 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:08.381971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43aeed7-8a18-4d93-8943-915c8e1b7ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:08.382529 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43aeed7-8a18-4d93-8943-915c8e1b7ed5 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:08.382802 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43aeed7-8a18-4d93-8943-915c8e1b7ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:08.383127 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43aeed7-8a18-4d93-8943-915c8e1b7ed5 None admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:08.383824 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43aeed7-8a18-4d93-8943-915c8e1b7ed5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:08.384171 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43aeed7-8a18-4d93-8943-915c8e1b7ed5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:08.385535 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a8f6a-c2e4-49c3-9ccf-713f3c99a1c3 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:08.385654 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43aeed7-8a18-4d93-8943-915c8e1b7ed5 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:08.399243 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 341/684] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:45:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:08.399441 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:08.400376 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9b73c-0194-438f-b467-87c99d59513a None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:08.400654 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9b73c-0194-438f-b467-87c99d59513a None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:08.400920 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9b73c-0194-438f-b467-87c99d59513a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:08.421817 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-2bb9b73c-0194-438f-b467-87c99d59513a None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:08.423094 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-2bb9b73c-0194-438f-b467-87c99d59513a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:08.429658 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6615708-5255-4078-9e5d-18542a6c188e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:08.430281 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6615708-5255-4078-9e5d-18542a6c188e None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:08.430589 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6615708-5255-4078-9e5d-18542a6c188e None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:08.430858 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6615708-5255-4078-9e5d-18542a6c188e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:08.487970 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:08.521205 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 342/685] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:45:08.522040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6615708-5255-4078-9e5d-18542a6c188e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:08.522512 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9b73c-0194-438f-b467-87c99d59513a None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:08.524214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6615708-5255-4078-9e5d-18542a6c188e None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:08.672316 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 344/686] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:45:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:45:08.678800 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9b73c-0194-438f-b467-87c99d59513a None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:08.687910 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6615708-5255-4078-9e5d-18542a6c188e None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:08.702841 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:08.735580 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361130d-ac81-4b34-8223-e1bf4ab7c99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:08.736194 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361130d-ac81-4b34-8223-e1bf4ab7c99a None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:08.736513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361130d-ac81-4b34-8223-e1bf4ab7c99a None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:08.736784 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361130d-ac81-4b34-8223-e1bf4ab7c99a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:08.737515 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f361130d-ac81-4b34-8223-e1bf4ab7c99a None admin] RBAC: Adding query filter params (role.id=bec755d6565a41798b09f5f57c1d51dd, scope.project.id=8db9b3f8a9f041938ac7b39003a35015, user.id=1a00b7f45b2e47b8a01048c36d00f742) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:08.737815 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f361130d-ac81-4b34-8223-e1bf4ab7c99a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:08.739274 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f361130d-ac81-4b34-8223-e1bf4ab7c99a None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:08.747211 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 343/687] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:45:08 2022] GET /identity/v3/role_assignments?user.id=1a00b7f45b2e47b8a01048c36d00f742&scope.project.id=8db9b3f8a9f041938ac7b39003a35015&role.id=bec755d6565a41798b09f5f57c1d51dd => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:08.832939 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361130d-ac81-4b34-8223-e1bf4ab7c99a None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:08.834363 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6615708-5255-4078-9e5d-18542a6c188e None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:08.884916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9b025-0b0c-4e47-adaf-2e9bd1eeab27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:08.885960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9b025-0b0c-4e47-adaf-2e9bd1eeab27 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:08.886338 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9b025-0b0c-4e47-adaf-2e9bd1eeab27 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:08.886923 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9b025-0b0c-4e47-adaf-2e9bd1eeab27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:08.900683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e9b025-0b0c-4e47-adaf-2e9bd1eeab27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:08.903029 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e9b025-0b0c-4e47-adaf-2e9bd1eeab27 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:08.924965 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-84e9b025-0b0c-4e47-adaf-2e9bd1eeab27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:08.924965 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:08.924965 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:08.924965 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:08.924965 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:08.924965 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:08.924965 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:08.924965 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:08.924965 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:08.924965 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:08.924965 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:08.924965 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:08.924965 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:08.924965 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:08.924965 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:08.924965 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:08.926139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:08.926139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:08.926139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:08.926139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:08.926139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:08.926139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:08.926139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:08.926139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:08.926139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:08.926139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:08.926139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:08.926139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:08.926139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:08.926139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:08.926139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:08.926139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:08.926139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:08.927203 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:08.927203 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:08.927203 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:08.930417 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 345/688] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:08.939544 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9b025-0b0c-4e47-adaf-2e9bd1eeab27 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:08.953797 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:09.024853 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:09.032020 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361130d-ac81-4b34-8223-e1bf4ab7c99a None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:09.080113 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804619bd-7f27-442a-89ac-ef86522cf3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:09.080816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804619bd-7f27-442a-89ac-ef86522cf3e8 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:09.081096 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804619bd-7f27-442a-89ac-ef86522cf3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:09.081364 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804619bd-7f27-442a-89ac-ef86522cf3e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:09.081540 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9b025-0b0c-4e47-adaf-2e9bd1eeab27 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:09.095691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804619bd-7f27-442a-89ac-ef86522cf3e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:45:09.099664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-804619bd-7f27-442a-89ac-ef86522cf3e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:45:09.099664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:09.099664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:09.099664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:09.099664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:09.099664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:09.099664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:09.099664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:09.099664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:09.099664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:09.099664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:09.099664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:09.099664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:45:09.099664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:45:09.099664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:45:09.099664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:45:09.101776 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:45:09.101776 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:45:09.101776 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:45:09.101776 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:45:09.101776 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:09.101776 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:09.101776 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:09.101776 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:09.101776 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:09.101776 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:09.101776 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:09.101776 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:09.101776 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:09.101776 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:09.101776 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:09.101776 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:09.102918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:09.102918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:09.102918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:09.102918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:09.102918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:45:09.102918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:45:09.102918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:45:09.102918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:45:09.102918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:45:09.102918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:45:09.102918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:45:09.102918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:09.102918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 344/689] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:45:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:09.107785 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804619bd-7f27-442a-89ac-ef86522cf3e8 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:09.135843 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e04724-a29b-41f1-a1b4-4df99120f952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:09.136462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e04724-a29b-41f1-a1b4-4df99120f952 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:09.136740 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e04724-a29b-41f1-a1b4-4df99120f952 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:09.137184 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e04724-a29b-41f1-a1b4-4df99120f952 None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:09.138143 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e04724-a29b-41f1-a1b4-4df99120f952 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:09.138504 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e04724-a29b-41f1-a1b4-4df99120f952 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:09.139972 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e04724-a29b-41f1-a1b4-4df99120f952 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:09.153070 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 346/690] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:09.160207 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e04724-a29b-41f1-a1b4-4df99120f952 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:09.223022 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804619bd-7f27-442a-89ac-ef86522cf3e8 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:09.224572 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:09.270815 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e04724-a29b-41f1-a1b4-4df99120f952 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:09.273896 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a38ac-38bf-4a1f-9d6d-dc83a9c64e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:09.274659 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a38ac-38bf-4a1f-9d6d-dc83a9c64e29 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:09.274933 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a38ac-38bf-4a1f-9d6d-dc83a9c64e29 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:09.275292 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a38ac-38bf-4a1f-9d6d-dc83a9c64e29 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:09.275972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a38ac-38bf-4a1f-9d6d-dc83a9c64e29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:09.276271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a38ac-38bf-4a1f-9d6d-dc83a9c64e29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:09.277732 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a38ac-38bf-4a1f-9d6d-dc83a9c64e29 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:09.294791 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:09.296523 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 345/691] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:45:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:09.302880 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a38ac-38bf-4a1f-9d6d-dc83a9c64e29 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:09.325031 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7f2ab-57ee-4b81-a637-514655e97c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:09.325832 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7f2ab-57ee-4b81-a637-514655e97c97 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:09.326111 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7f2ab-57ee-4b81-a637-514655e97c97 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:09.326384 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7f2ab-57ee-4b81-a637-514655e97c97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:09.337925 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7f2ab-57ee-4b81-a637-514655e97c97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:09.340295 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7f2ab-57ee-4b81-a637-514655e97c97 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:09.360870 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-14d7f2ab-57ee-4b81-a637-514655e97c97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:09.360870 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:09.360870 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:09.360870 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:09.360870 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:09.360870 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:09.360870 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:09.360870 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:09.360870 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:09.360870 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:09.360870 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:09.360870 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:09.360870 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:45:09.360870 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:45:09.360870 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:45:09.363058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:45:09.363058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:09.363058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:09.363058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:09.363058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:09.363058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:09.363058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:09.363058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:09.363058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:09.363058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:09.363058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:09.363058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:09.363058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:09.363058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:09.363058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:09.363058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:09.363058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:09.365192 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:45:09.365192 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:45:09.365192 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:45:09.365192 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:45:09.365192 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:45:09.365192 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:45:09.365192 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:09.365192 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:09.365192 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 347/692] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:45:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:09.371917 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7f2ab-57ee-4b81-a637-514655e97c97 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:09.450343 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a38ac-38bf-4a1f-9d6d-dc83a9c64e29 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:09.488862 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e062f38-5784-4e40-9d5c-6e7572b5ecaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:09.489477 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e062f38-5784-4e40-9d5c-6e7572b5ecaf None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:09.489748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e062f38-5784-4e40-9d5c-6e7572b5ecaf None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:09.490016 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e062f38-5784-4e40-9d5c-6e7572b5ecaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:09.510510 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e062f38-5784-4e40-9d5c-6e7572b5ecaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:09.512141 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e062f38-5784-4e40-9d5c-6e7572b5ecaf None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:09.537915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-0e062f38-5784-4e40-9d5c-6e7572b5ecaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:09.537915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:09.537915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:09.537915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:09.537915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:09.537915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:09.537915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:09.537915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:09.537915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:09.537915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:09.537915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:09.537915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:09.537915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:09.537915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:09.537915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:09.537915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:09.540173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:09.540173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:09.540173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:09.540173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:09.540173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:09.540173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:09.540173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:09.540173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:09.540173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:09.540173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:09.540173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:09.540173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:09.540173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:09.540173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:09.540173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:09.540173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:09.540173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:09.542332 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:09.542332 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:09.542332 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:09.542332 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 346/693] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:09.550539 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e062f38-5784-4e40-9d5c-6e7572b5ecaf None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:09.589704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:09.600326 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7f2ab-57ee-4b81-a637-514655e97c97 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:09.658222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f472e7cc-7a8f-4dc6-9ef1-ba11b4d7474d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:09.659105 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f472e7cc-7a8f-4dc6-9ef1-ba11b4d7474d None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:09.659541 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f472e7cc-7a8f-4dc6-9ef1-ba11b4d7474d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:09.660132 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f472e7cc-7a8f-4dc6-9ef1-ba11b4d7474d None admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:09.661622 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f472e7cc-7a8f-4dc6-9ef1-ba11b4d7474d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:09.662105 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f472e7cc-7a8f-4dc6-9ef1-ba11b4d7474d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:09.664351 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f472e7cc-7a8f-4dc6-9ef1-ba11b4d7474d None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:09.682922 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 348/694] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:45:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:09.684676 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96707963-2c41-485f-8bd0-85740793c1a6 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:09.685230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96707963-2c41-485f-8bd0-85740793c1a6 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:09.685726 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96707963-2c41-485f-8bd0-85740793c1a6 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:09.691338 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 349/695] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:09.692464 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96707963-2c41-485f-8bd0-85740793c1a6 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:09.736196 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e062f38-5784-4e40-9d5c-6e7572b5ecaf None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:09.747366 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:09.778859 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ff3dc-763e-4757-9596-a869935cc4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:09.779554 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ff3dc-763e-4757-9596-a869935cc4a4 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:09.779834 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ff3dc-763e-4757-9596-a869935cc4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:09.780102 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ff3dc-763e-4757-9596-a869935cc4a4 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:09.780755 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ff3dc-763e-4757-9596-a869935cc4a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:09.781113 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ff3dc-763e-4757-9596-a869935cc4a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:09.783318 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ff3dc-763e-4757-9596-a869935cc4a4 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:09.797219 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 347/696] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:09.798772 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559510a-d305-45da-94ce-6327065ed84a None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:09.798995 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559510a-d305-45da-94ce-6327065ed84a None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:09.799339 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559510a-d305-45da-94ce-6327065ed84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:09.819072 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-c559510a-d305-45da-94ce-6327065ed84a None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:09.820652 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-c559510a-d305-45da-94ce-6327065ed84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:09.832107 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96707963-2c41-485f-8bd0-85740793c1a6 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:09.882752 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fbebbb-2fe2-456d-9b5c-ecf9a5ec1698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:09.884296 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbebbb-2fe2-456d-9b5c-ecf9a5ec1698 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:09.885063 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbebbb-2fe2-456d-9b5c-ecf9a5ec1698 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:09.885544 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbebbb-2fe2-456d-9b5c-ecf9a5ec1698 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:09.886835 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fbebbb-2fe2-456d-9b5c-ecf9a5ec1698 None admin] RBAC: Adding query filter params (role.id=bec755d6565a41798b09f5f57c1d51dd, scope.project.id=8db9b3f8a9f041938ac7b39003a35015, user.id=e8abc7c439a04622896b34ed0a7b8492) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:09.887826 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fbebbb-2fe2-456d-9b5c-ecf9a5ec1698 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:09.890047 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fbebbb-2fe2-456d-9b5c-ecf9a5ec1698 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:09.898832 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 350/697] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:45:09 2022] GET /identity/v3/role_assignments?user.id=e8abc7c439a04622896b34ed0a7b8492&scope.project.id=8db9b3f8a9f041938ac7b39003a35015&role.id=bec755d6565a41798b09f5f57c1d51dd => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:09.899963 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fbebbb-2fe2-456d-9b5c-ecf9a5ec1698 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:09.912165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 348/698] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:45:10.080596 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fbebbb-2fe2-456d-9b5c-ecf9a5ec1698 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:10.083113 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:10.123803 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e92389-675d-4b44-84a0-727feb1be24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:10.124446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e92389-675d-4b44-84a0-727feb1be24f None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:10.124949 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e92389-675d-4b44-84a0-727feb1be24f None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:10.125808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e92389-675d-4b44-84a0-727feb1be24f None admin] PATH_INFO: `/v3/users/nova` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:10.135117 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e92389-675d-4b44-84a0-727feb1be24f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:10.136570 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e92389-675d-4b44-84a0-727feb1be24f None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:10.154285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-75e92389-675d-4b44-84a0-727feb1be24f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:45:10.154285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:10.154285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:10.154285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:10.154285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:10.154285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:10.154285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:10.154285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:10.154285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:10.154285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:10.154285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:10.154285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:10.154285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:45:10.154285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:45:10.154285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:45:10.154285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:45:10.155322 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:10.155322 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:10.155322 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:45:10.155322 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:10.155322 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:45:10.155322 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:10.155322 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:10.155322 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:10.155322 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:10.155322 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:10.155322 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:10.155322 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:10.155322 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:10.155322 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:10.155322 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:10.155322 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:10.155322 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:10.155322 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:10.156264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:10.156264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:10.156264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:45:10.156264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:45:10.156264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:45:10.156264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:45:10.156264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:45:10.156264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:45:10.156264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:45:10.156264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:10.156264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 351/699] 10.222.0.16 () {56 vars in 1093 bytes} [Wed Aug 31 10:45:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:10.161252 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559510a-d305-45da-94ce-6327065ed84a None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:10.172259 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:10.281695 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559510a-d305-45da-94ce-6327065ed84a None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:10.290589 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e92389-675d-4b44-84a0-727feb1be24f None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:10.327090 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef2efc-8fa0-420e-b09a-d4bb27530259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:10.327991 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef2efc-8fa0-420e-b09a-d4bb27530259 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:10.328269 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef2efc-8fa0-420e-b09a-d4bb27530259 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:10.328540 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef2efc-8fa0-420e-b09a-d4bb27530259 None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:10.329555 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef2efc-8fa0-420e-b09a-d4bb27530259 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:10.329863 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef2efc-8fa0-420e-b09a-d4bb27530259 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:10.332114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef2efc-8fa0-420e-b09a-d4bb27530259 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:10.356691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 349/700] 10.222.0.16 () {56 vars in 1133 bytes} [Wed Aug 31 10:45:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:10.362240 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef2efc-8fa0-420e-b09a-d4bb27530259 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:10.428426 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e92389-675d-4b44-84a0-727feb1be24f None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:10.442628 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:10.466172 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5032d553-0211-45ed-8b52-f591ace70520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:10.466850 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032d553-0211-45ed-8b52-f591ace70520 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:10.473457 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032d553-0211-45ed-8b52-f591ace70520 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:10.473457 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032d553-0211-45ed-8b52-f591ace70520 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:10.473457 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5032d553-0211-45ed-8b52-f591ace70520 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:10.473457 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5032d553-0211-45ed-8b52-f591ace70520 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:10.482465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-5032d553-0211-45ed-8b52-f591ace70520 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 10:45:10.482465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:10.482465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:45:10.482465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:45:10.482465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:45:10.482465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:45:10.482465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:45:10.482465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:10.482465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 10:45:10.482465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:10.482465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:45:10.482465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:10.482465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:10.482465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:10.482465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:10.482465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:10.482465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:10.483588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:10.483588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:10.483588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:10.483588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:10.483588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:10.483588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:10.483588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:45:10.483588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:45:10.483588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:45:10.483588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:45:10.483588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:10.483588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:10.483588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:10.483588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:10.483588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:10.483588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:10.484423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:10.484423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:10.484423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:10.484423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:10.484423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:10.484423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:10.484423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:10.484423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:10.484423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:10.484423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:10.484423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:45:10.484423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:10.484423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 10:45:10.484423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:10.484423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 352/701] 10.222.0.16 () {56 vars in 1129 bytes} [Wed Aug 31 10:45:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:10.491182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5032d553-0211-45ed-8b52-f591ace70520 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:10.496051 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef2efc-8fa0-420e-b09a-d4bb27530259 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:10.534880 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206e4fe-7094-493e-865c-0574245431d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:10.535954 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206e4fe-7094-493e-865c-0574245431d0 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:10.536547 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206e4fe-7094-493e-865c-0574245431d0 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:10.537048 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206e4fe-7094-493e-865c-0574245431d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:10.540672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:10.636624 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206e4fe-7094-493e-865c-0574245431d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:10.639417 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206e4fe-7094-493e-865c-0574245431d0 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:10.643332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5032d553-0211-45ed-8b52-f591ace70520 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:10.678442 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c29b7-c848-4265-a2f7-f260cb176f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:10.679052 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c29b7-c848-4265-a2f7-f260cb176f0f None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:10.679334 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c29b7-c848-4265-a2f7-f260cb176f0f None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:10.679606 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c29b7-c848-4265-a2f7-f260cb176f0f None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:10.680236 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c29b7-c848-4265-a2f7-f260cb176f0f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:10.680534 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c29b7-c848-4265-a2f7-f260cb176f0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:10.681911 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c29b7-c848-4265-a2f7-f260cb176f0f None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:10.687537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 353/702] 10.222.0.16 () {56 vars in 1124 bytes} [Wed Aug 31 10:45:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:10.688676 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d1242-914b-4f7f-8877-699cbb356d48 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:10.688957 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d1242-914b-4f7f-8877-699cbb356d48 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:10.689227 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d1242-914b-4f7f-8877-699cbb356d48 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:10.694832 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 354/703] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:10.695954 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa771636-3ded-4514-bc2d-6547a99752bb None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:10.696239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa771636-3ded-4514-bc2d-6547a99752bb None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:10.696513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa771636-3ded-4514-bc2d-6547a99752bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:10.712103 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:10.715534 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-fa771636-3ded-4514-bc2d-6547a99752bb None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:10.716737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-fa771636-3ded-4514-bc2d-6547a99752bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:10.762860 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 350/704] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:45:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:45:10.764801 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206e4fe-7094-493e-865c-0574245431d0 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:10.794292 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 355/705] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:45:10.798902 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa771636-3ded-4514-bc2d-6547a99752bb None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:10.841343 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:10.887830 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206e4fe-7094-493e-865c-0574245431d0 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:10.946421 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa771636-3ded-4514-bc2d-6547a99752bb None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:10.978833 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b773b7-9654-4295-959e-d3fd91724267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:10.979494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b773b7-9654-4295-959e-d3fd91724267 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:10.979771 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b773b7-9654-4295-959e-d3fd91724267 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:10.980040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b773b7-9654-4295-959e-d3fd91724267 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:10.984063 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755460c-f149-4c43-8379-52659c664993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:10.984696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755460c-f149-4c43-8379-52659c664993 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:10.984976 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755460c-f149-4c43-8379-52659c664993 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:10.985249 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755460c-f149-4c43-8379-52659c664993 None admin] PATH_INFO: `/v3/services` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:10.985897 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755460c-f149-4c43-8379-52659c664993 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:10.986201 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755460c-f149-4c43-8379-52659c664993 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:10.987609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755460c-f149-4c43-8379-52659c664993 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:10.992448 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b773b7-9654-4295-959e-d3fd91724267 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:10.993926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b773b7-9654-4295-959e-d3fd91724267 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:11.010334 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-d7b773b7-9654-4295-959e-d3fd91724267 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:11.010334 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:11.010334 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:11.010334 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:11.010334 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:11.010334 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:11.010334 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:11.010334 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:11.010334 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:11.010334 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:11.010334 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:11.010334 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:11.010334 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:11.010334 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:11.010334 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:11.010334 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:11.011386 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:11.011386 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:11.011386 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:11.011386 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:11.011386 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:11.011386 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:11.011386 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:11.011386 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:11.011386 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:11.011386 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:11.011386 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:11.011386 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:11.011386 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:11.011386 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:11.011386 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:11.011386 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:11.011386 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:11.012510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:11.012510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:11.012510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:11.012510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 356/706] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:11.012744 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 351/707] 10.222.0.16 () {56 vars in 1124 bytes} [Wed Aug 31 10:45:10 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:11.018818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b773b7-9654-4295-959e-d3fd91724267 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:11.186768 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:11.214028 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b773b7-9654-4295-959e-d3fd91724267 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:11.253538 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5fc17a-d96d-40a2-a0ed-e342188b1290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:11.254163 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fc17a-d96d-40a2-a0ed-e342188b1290 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:11.254488 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fc17a-d96d-40a2-a0ed-e342188b1290 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:11.254771 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fc17a-d96d-40a2-a0ed-e342188b1290 None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:11.255432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5fc17a-d96d-40a2-a0ed-e342188b1290 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:11.255740 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5fc17a-d96d-40a2-a0ed-e342188b1290 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:11.257704 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5fc17a-d96d-40a2-a0ed-e342188b1290 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:11.269589 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755460c-f149-4c43-8379-52659c664993 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:11.274285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 357/708] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:11.284706 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5fc17a-d96d-40a2-a0ed-e342188b1290 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:11.312223 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:11.391750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755460c-f149-4c43-8379-52659c664993 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:11.445596 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a93301-408b-4637-ab9a-a83cffe33c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:11.446149 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a93301-408b-4637-ab9a-a83cffe33c1f None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:11.446477 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a93301-408b-4637-ab9a-a83cffe33c1f None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:11.446722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a93301-408b-4637-ab9a-a83cffe33c1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:11.449091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5fc17a-d96d-40a2-a0ed-e342188b1290 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:11.458635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a93301-408b-4637-ab9a-a83cffe33c1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:11.460415 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a93301-408b-4637-ab9a-a83cffe33c1f None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:11.477618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-e3a93301-408b-4637-ab9a-a83cffe33c1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:11.477618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:11.477618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:11.477618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:11.477618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:11.477618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:11.477618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:11.477618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:11.477618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:11.477618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:11.477618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:11.477618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:11.477618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:11.477618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:11.477618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:11.477618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:11.478678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:11.478678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:11.478678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:11.478678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:11.478678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:11.478678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:11.478678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:11.478678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:11.478678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:11.478678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:11.478678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:11.478678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:11.478678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:11.478678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:11.478678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:11.478678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:11.478678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:11.479556 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:11.479556 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:11.479556 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:11.479556 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 352/709] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:11.480394 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa18cfc-1be1-42f7-b802-7f2691fac716 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:11.480820 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa18cfc-1be1-42f7-b802-7f2691fac716 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:11.481277 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa18cfc-1be1-42f7-b802-7f2691fac716 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:11.487214 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 353/710] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:11.488123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa18cfc-1be1-42f7-b802-7f2691fac716 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:11.492596 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679c172-9da3-49f1-a0b0-52c0cd86fa39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:11.493321 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679c172-9da3-49f1-a0b0-52c0cd86fa39 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:11.493655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679c172-9da3-49f1-a0b0-52c0cd86fa39 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:11.494023 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679c172-9da3-49f1-a0b0-52c0cd86fa39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:11.496836 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:11.503490 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679c172-9da3-49f1-a0b0-52c0cd86fa39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:11.505452 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679c172-9da3-49f1-a0b0-52c0cd86fa39 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:11.519626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-d679c172-9da3-49f1-a0b0-52c0cd86fa39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:11.519626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:11.519626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:11.519626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:11.519626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:11.519626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:11.519626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:11.519626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:11.519626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:11.519626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:11.519626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:11.519626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:11.519626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:45:11.519626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:45:11.519626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:45:11.520610 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:45:11.520610 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:11.520610 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:11.520610 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:11.520610 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:11.520610 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:11.520610 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:11.520610 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:11.520610 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:11.520610 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:11.520610 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:11.520610 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:11.520610 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:11.520610 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:11.520610 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:11.520610 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:11.520610 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:11.521527 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:45:11.521527 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:45:11.521527 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:45:11.521527 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:45:11.521527 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:45:11.521527 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:45:11.521527 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:11.521527 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:11.521527 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 358/711] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:45:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:11.528689 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c28ee-3e95-43c6-8056-e7536b06a535 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:11.528985 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c28ee-3e95-43c6-8056-e7536b06a535 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:11.529184 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c28ee-3e95-43c6-8056-e7536b06a535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:11.550091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-423c28ee-3e95-43c6-8056-e7536b06a535 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:11.551324 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-423c28ee-3e95-43c6-8056-e7536b06a535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:11.608918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:11.634416 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 359/712] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:45:11.634416 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c28ee-3e95-43c6-8056-e7536b06a535 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:11.653494 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa18cfc-1be1-42f7-b802-7f2691fac716 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:11.727230 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f1bc2-cc9c-413f-94c6-5fe0ef7b2503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:11.727781 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f1bc2-cc9c-413f-94c6-5fe0ef7b2503 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:11.728058 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f1bc2-cc9c-413f-94c6-5fe0ef7b2503 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:11.728333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f1bc2-cc9c-413f-94c6-5fe0ef7b2503 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:11.728996 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f1bc2-cc9c-413f-94c6-5fe0ef7b2503 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:11.729302 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f1bc2-cc9c-413f-94c6-5fe0ef7b2503 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:11.738439 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f1bc2-cc9c-413f-94c6-5fe0ef7b2503 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:11.744873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 354/713] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:11.751165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f1bc2-cc9c-413f-94c6-5fe0ef7b2503 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:11.809289 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c28ee-3e95-43c6-8056-e7536b06a535 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:11.857642 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:11.857837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f1bc2-cc9c-413f-94c6-5fe0ef7b2503 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:11.877523 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ed11f-e12b-46db-ac55-61e8d021a76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:11.879093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ed11f-e12b-46db-ac55-61e8d021a76a None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:11.879093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ed11f-e12b-46db-ac55-61e8d021a76a None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:11.879093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ed11f-e12b-46db-ac55-61e8d021a76a None admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:11.884044 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ed11f-e12b-46db-ac55-61e8d021a76a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:11.884411 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ed11f-e12b-46db-ac55-61e8d021a76a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:11.885978 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ed11f-e12b-46db-ac55-61e8d021a76a None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:11.901814 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c050e-7b70-4ab1-8779-ab671cad1621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:11.902539 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c050e-7b70-4ab1-8779-ab671cad1621 None admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:11.902827 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c050e-7b70-4ab1-8779-ab671cad1621 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:11.903087 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c050e-7b70-4ab1-8779-ab671cad1621 None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:11.903874 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c050e-7b70-4ab1-8779-ab671cad1621 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:11.905323 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c050e-7b70-4ab1-8779-ab671cad1621 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:11.916996 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:11.917202 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 360/714] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:45:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:11.924098 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ed11f-e12b-46db-ac55-61e8d021a76a None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:12.018633 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 355/715] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 31 10:45:11 2022] POST /identity/v3/users => generated 254 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 10:45:12.019820 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c050e-7b70-4ab1-8779-ab671cad1621 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:12.064390 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ed11f-e12b-46db-ac55-61e8d021a76a None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:12.118500 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fad68-0076-4da0-a1f7-e852c1847239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:12.119788 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fad68-0076-4da0-a1f7-e852c1847239 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:12.120273 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fad68-0076-4da0-a1f7-e852c1847239 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:12.120746 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fad68-0076-4da0-a1f7-e852c1847239 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:12.122821 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8fad68-0076-4da0-a1f7-e852c1847239 None admin] RBAC: Adding query filter params (role.id=13ca33d694c645d6920930d39dc52a0d, scope.project.id=8db9b3f8a9f041938ac7b39003a35015, user.id=f11971a594014d86968c3356d11d42f7) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:12.123415 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8fad68-0076-4da0-a1f7-e852c1847239 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:12.125121 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8fad68-0076-4da0-a1f7-e852c1847239 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:12.132239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 361/716] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:45:11 2022] GET /identity/v3/role_assignments?user.id=f11971a594014d86968c3356d11d42f7&scope.project.id=8db9b3f8a9f041938ac7b39003a35015&role.id=13ca33d694c645d6920930d39dc52a0d => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:12.219744 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c050e-7b70-4ab1-8779-ab671cad1621 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:12.276154 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf9886-1dfd-4f7a-8273-df215d469f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:12.278145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf9886-1dfd-4f7a-8273-df215d469f35 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:12.278681 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf9886-1dfd-4f7a-8273-df215d469f35 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:12.279158 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf9886-1dfd-4f7a-8273-df215d469f35 None admin] PATH_INFO: `/v3/services/network` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:12.280008 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf9886-1dfd-4f7a-8273-df215d469f35 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:12.281801 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf9886-1dfd-4f7a-8273-df215d469f35 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:12.299436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-efcf9886-1dfd-4f7a-8273-df215d469f35 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 10:45:12.299436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:12.299436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:45:12.299436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:45:12.299436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:45:12.299436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:45:12.299436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:45:12.299436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:12.299436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 10:45:12.299436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:12.299436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:45:12.299436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:12.299436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:12.299436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:12.299436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:12.299436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:12.299436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:12.300489 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:12.300489 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:12.300489 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:12.300489 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:12.300489 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:12.300489 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:12.300489 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:45:12.300489 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:45:12.300489 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:45:12.300489 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:45:12.300489 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:12.300489 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:12.300489 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:12.300489 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:12.300489 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:12.300489 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:12.301314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:12.301314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:12.301314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:12.301314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:12.301314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:12.301314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:12.301314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:12.301314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:12.301314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:12.301314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:12.301314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:45:12.301314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:12.301314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 10:45:12.301314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:12.303486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 356/717] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:45:12 2022] GET /identity/v3/services/network => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:12.311178 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fad68-0076-4da0-a1f7-e852c1847239 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:12.369611 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a0141-85b3-4bc4-979a-54f1a95f04d9 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:12.369886 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a0141-85b3-4bc4-979a-54f1a95f04d9 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:12.370158 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a0141-85b3-4bc4-979a-54f1a95f04d9 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:12.370468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:12.373111 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 357/718] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:12.377537 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fd368-3b5d-498b-bf35-f547c268467a None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:12.377781 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fd368-3b5d-498b-bf35-f547c268467a None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:12.378146 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fd368-3b5d-498b-bf35-f547c268467a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:12.398537 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-5f1fd368-3b5d-498b-bf35-f547c268467a None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:12.399746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-5f1fd368-3b5d-498b-bf35-f547c268467a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:12.434930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fad68-0076-4da0-a1f7-e852c1847239 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:12.469506 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:12.472575 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d691e-b474-4617-8514-1291e95d9e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:12.473196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d691e-b474-4617-8514-1291e95d9e23 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:12.473527 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d691e-b474-4617-8514-1291e95d9e23 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:12.473799 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d691e-b474-4617-8514-1291e95d9e23 None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:12.474515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d691e-b474-4617-8514-1291e95d9e23 None admin] RBAC: Adding query filter params (name=network) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:12.474819 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d691e-b474-4617-8514-1291e95d9e23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:12.476309 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d691e-b474-4617-8514-1291e95d9e23 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:12.482428 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 362/719] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Aug 31 10:45:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:12.486903 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 358/720] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:45:12.492931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d691e-b474-4617-8514-1291e95d9e23 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:12.620905 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d691e-b474-4617-8514-1291e95d9e23 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:12.635511 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:12.659769 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6f156-6509-4820-be27-f11d8d03becc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:12.660841 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6f156-6509-4820-be27-f11d8d03becc None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:12.661344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6f156-6509-4820-be27-f11d8d03becc None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:12.661824 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6f156-6509-4820-be27-f11d8d03becc None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:12.662711 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6f156-6509-4820-be27-f11d8d03becc None admin] RBAC: Adding query filter params (type=network) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:12.663246 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6f156-6509-4820-be27-f11d8d03becc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:12.665008 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6f156-6509-4820-be27-f11d8d03becc None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:12.673064 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 363/721] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Aug 31 10:45:12 2022] GET /identity/v3/services?type=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:12.802251 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1fd368-3b5d-498b-bf35-f547c268467a None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:12.945276 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1fd368-3b5d-498b-bf35-f547c268467a None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:12.986589 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56e636-56f4-4824-8ef6-59c1e6d9c2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:12.987378 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56e636-56f4-4824-8ef6-59c1e6d9c2c5 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:12.987757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56e636-56f4-4824-8ef6-59c1e6d9c2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:12.988129 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56e636-56f4-4824-8ef6-59c1e6d9c2c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:12.999618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b56e636-56f4-4824-8ef6-59c1e6d9c2c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:45:13.004605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-3b56e636-56f4-4824-8ef6-59c1e6d9c2c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:45:13.004605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:13.004605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:13.004605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:13.004605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:13.004605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:13.004605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:13.004605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:13.004605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:13.004605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:13.004605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:13.004605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:13.004605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:45:13.004605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:45:13.004605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:45:13.004605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:45:13.005722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:45:13.005722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:45:13.005722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:45:13.005722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:45:13.005722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:13.005722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:13.005722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:13.005722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:13.005722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:13.005722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:13.005722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:13.005722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:13.005722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:13.005722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:13.005722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:13.005722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:13.006760 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:13.006760 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:13.006760 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:13.006760 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:13.006760 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:45:13.006760 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:45:13.006760 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:45:13.006760 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:45:13.006760 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:45:13.006760 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:45:13.006760 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:45:13.006760 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:13.008669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 359/722] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:45:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:13.015162 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6f156-6509-4820-be27-f11d8d03becc None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:13.223485 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6f156-6509-4820-be27-f11d8d03becc None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:13.262677 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b731780e-8541-4653-bd45-86a423bd4fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:13.265070 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b731780e-8541-4653-bd45-86a423bd4fef None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:13.265070 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b731780e-8541-4653-bd45-86a423bd4fef None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:13.265070 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b731780e-8541-4653-bd45-86a423bd4fef None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:13.266660 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b731780e-8541-4653-bd45-86a423bd4fef None admin] RBAC: Adding query filter params (name=admin) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:13.267000 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b731780e-8541-4653-bd45-86a423bd4fef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:13.268420 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b731780e-8541-4653-bd45-86a423bd4fef None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:13.285787 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 364/723] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:45:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:13.291226 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56e636-56f4-4824-8ef6-59c1e6d9c2c5 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:13.434466 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:13.508609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56e636-56f4-4824-8ef6-59c1e6d9c2c5 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:13.519922 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dea45-6de3-4922-83d7-01f7cb890a02 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:13.520222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dea45-6de3-4922-83d7-01f7cb890a02 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:13.520926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dea45-6de3-4922-83d7-01f7cb890a02 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:13.523352 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 365/724] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:13.532617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77a3e2-1a30-4e2c-97b7-cd5910f2d955 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:13.535991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77a3e2-1a30-4e2c-97b7-cd5910f2d955 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:13.536302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77a3e2-1a30-4e2c-97b7-cd5910f2d955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:13.572124 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953afef-c918-47a9-a9c7-0721de04f145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:13.572769 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953afef-c918-47a9-a9c7-0721de04f145 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:13.573040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953afef-c918-47a9-a9c7-0721de04f145 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:13.573305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953afef-c918-47a9-a9c7-0721de04f145 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:13.596163 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953afef-c918-47a9-a9c7-0721de04f145 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:13.597633 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953afef-c918-47a9-a9c7-0721de04f145 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:13.598936 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-cc77a3e2-1a30-4e2c-97b7-cd5910f2d955 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:13.600145 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-cc77a3e2-1a30-4e2c-97b7-cd5910f2d955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:13.628093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-8953afef-c918-47a9-a9c7-0721de04f145 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:13.628093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:13.628093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:13.628093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:13.628093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:13.628093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:13.628093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:13.628093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:13.628093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:13.628093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:13.628093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:13.628093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:13.628093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:13.628093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:13.628093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:13.628093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:13.629085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:13.629085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:13.629085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:13.629085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:13.629085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:13.629085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:13.629085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:13.629085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:13.629085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:13.629085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:13.629085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:13.629085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:13.629085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:13.629085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:13.629085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:13.629085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:13.629085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:13.629952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:13.629952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:13.629952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:13.634213 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 360/725] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:13.641358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953afef-c918-47a9-a9c7-0721de04f145 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:13.680307 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:13.771201 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 366/726] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:45:13.861661 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953afef-c918-47a9-a9c7-0721de04f145 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:13.874797 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:13.911406 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6d0b9-a276-4619-bf9e-daa23eaeef64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:13.912020 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6d0b9-a276-4619-bf9e-daa23eaeef64 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:13.912297 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6d0b9-a276-4619-bf9e-daa23eaeef64 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:13.912571 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6d0b9-a276-4619-bf9e-daa23eaeef64 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:13.913217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6d0b9-a276-4619-bf9e-daa23eaeef64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:13.913519 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6d0b9-a276-4619-bf9e-daa23eaeef64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:13.914922 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6d0b9-a276-4619-bf9e-daa23eaeef64 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:13.934454 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 361/727] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:13.940411 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc77a3e2-1a30-4e2c-97b7-cd5910f2d955 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:14.090941 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc77a3e2-1a30-4e2c-97b7-cd5910f2d955 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:14.144804 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0295875-384c-48f3-9fb7-447f38c0e609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:14.145835 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0295875-384c-48f3-9fb7-447f38c0e609 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:14.146375 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0295875-384c-48f3-9fb7-447f38c0e609 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:14.146896 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0295875-384c-48f3-9fb7-447f38c0e609 None admin] PATH_INFO: `/v3/users/swift` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:14.148972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6d0b9-a276-4619-bf9e-daa23eaeef64 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:14.162526 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0295875-384c-48f3-9fb7-447f38c0e609 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:14.165715 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0295875-384c-48f3-9fb7-447f38c0e609 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:14.178953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-b0295875-384c-48f3-9fb7-447f38c0e609 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 10:45:14.178953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:14.178953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:14.178953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:14.178953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:14.178953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:14.178953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:14.178953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:14.178953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:14.178953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:14.178953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:14.178953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:14.178953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:45:14.178953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:45:14.178953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:45:14.178953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:45:14.179955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:14.179955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:14.179955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:45:14.179955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:14.179955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:45:14.179955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:14.179955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:14.179955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:14.179955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:14.179955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:14.179955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:14.179955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:14.179955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:14.179955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:14.179955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:14.179955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:14.179955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:14.179955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:14.180955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:14.180955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:14.180955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:45:14.180955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:45:14.180955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:45:14.180955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:45:14.180955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:45:14.180955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:45:14.180955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 10:45:14.180955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:14.184380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 367/728] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:45:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:14.190114 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0295875-384c-48f3-9fb7-447f38c0e609 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:14.242872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:14.340899 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6d0b9-a276-4619-bf9e-daa23eaeef64 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:14.385704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d83b16-8a85-47a9-b363-1be871b3742a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:14.386238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d83b16-8a85-47a9-b363-1be871b3742a None admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:14.386546 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d83b16-8a85-47a9-b363-1be871b3742a None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:14.387447 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d83b16-8a85-47a9-b363-1be871b3742a None admin] PATH_INFO: `/v3/services` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:14.388224 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d83b16-8a85-47a9-b363-1be871b3742a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:14.389618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d83b16-8a85-47a9-b363-1be871b3742a None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:14.398573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0295875-384c-48f3-9fb7-447f38c0e609 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:14.436832 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd28f65-9d8d-424b-9d69-47c6de211cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:14.437807 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd28f65-9d8d-424b-9d69-47c6de211cdb None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:14.438313 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd28f65-9d8d-424b-9d69-47c6de211cdb None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:14.438807 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd28f65-9d8d-424b-9d69-47c6de211cdb None admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:14.439724 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd28f65-9d8d-424b-9d69-47c6de211cdb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:14.440244 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd28f65-9d8d-424b-9d69-47c6de211cdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:14.442021 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd28f65-9d8d-424b-9d69-47c6de211cdb None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:14.466438 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 368/729] 10.222.0.16 () {56 vars in 1135 bytes} [Wed Aug 31 10:45:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:14.468008 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd28f65-9d8d-424b-9d69-47c6de211cdb None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:14.497858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 362/730] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Aug 31 10:45:14 2022] POST /identity/v3/services => generated 251 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 10:45:14.565653 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:14.579965 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87989ab7-460a-4094-852a-7ecbc4ef65a1 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:14.580301 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87989ab7-460a-4094-852a-7ecbc4ef65a1 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:14.580525 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87989ab7-460a-4094-852a-7ecbc4ef65a1 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:14.582110 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 363/731] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:14.583306 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd28f65-9d8d-424b-9d69-47c6de211cdb None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:14.594619 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaec102-6e6a-489e-81c2-e1431d01b56b None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:14.594818 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaec102-6e6a-489e-81c2-e1431d01b56b None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:14.595289 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaec102-6e6a-489e-81c2-e1431d01b56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:14.636658 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23040ae5-5cff-440b-a2cc-d6cc3016a8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:14.637685 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23040ae5-5cff-440b-a2cc-d6cc3016a8a1 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:14.638217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23040ae5-5cff-440b-a2cc-d6cc3016a8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:14.638738 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23040ae5-5cff-440b-a2cc-d6cc3016a8a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:14.644750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-afaec102-6e6a-489e-81c2-e1431d01b56b None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:14.648207 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-afaec102-6e6a-489e-81c2-e1431d01b56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:14.672401 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:14.749435 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23040ae5-5cff-440b-a2cc-d6cc3016a8a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:14.751269 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23040ae5-5cff-440b-a2cc-d6cc3016a8a1 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:14.755222 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 364/732] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:14 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:45:14.841012 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:14.863746 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 369/733] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:45:14 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 10:45:14.869482 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaec102-6e6a-489e-81c2-e1431d01b56b None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:14.982198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaec102-6e6a-489e-81c2-e1431d01b56b None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:15.002626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ccff9-010e-4158-a7d2-210e7ebdd76b None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:15.002968 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ccff9-010e-4158-a7d2-210e7ebdd76b None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:15.003180 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ccff9-010e-4158-a7d2-210e7ebdd76b None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:15.004797 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 370/734] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:15.009980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f5395-5c9f-4a48-a155-c65ea86eab20 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:15.010265 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f5395-5c9f-4a48-a155-c65ea86eab20 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:15.010577 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f5395-5c9f-4a48-a155-c65ea86eab20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:15.036315 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd96613-81dd-4d68-b236-1a42f0aef2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:15.036969 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd96613-81dd-4d68-b236-1a42f0aef2bb None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:15.037249 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd96613-81dd-4d68-b236-1a42f0aef2bb None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:15.037518 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd96613-81dd-4d68-b236-1a42f0aef2bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:15.053883 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-158f5395-5c9f-4a48-a155-c65ea86eab20 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:15.055130 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-158f5395-5c9f-4a48-a155-c65ea86eab20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:15.059947 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd96613-81dd-4d68-b236-1a42f0aef2bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:15.061417 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd96613-81dd-4d68-b236-1a42f0aef2bb None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:15.093796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-3cd96613-81dd-4d68-b236-1a42f0aef2bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:15.093796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:15.093796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:15.093796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:15.093796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:15.093796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:15.093796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:15.093796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:15.093796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:15.093796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:15.093796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:15.093796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:15.093796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:15.093796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:15.093796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:15.093796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:15.094912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:15.094912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:15.094912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:15.094912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:15.094912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:15.094912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:15.094912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:15.094912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:15.094912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:15.094912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:15.094912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:15.094912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:15.094912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:15.094912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:15.094912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:15.094912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:15.094912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:15.095780 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:15.095780 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:15.095780 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:15.095780 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 365/735] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:15.095780 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd96613-81dd-4d68-b236-1a42f0aef2bb None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:15.100289 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:15.170780 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 371/736] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:15 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:45:15.171992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f5395-5c9f-4a48-a155-c65ea86eab20 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:15.226076 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd96613-81dd-4d68-b236-1a42f0aef2bb None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:15.268196 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cbb1e-1083-435a-a451-0a878153d4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:15.269113 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cbb1e-1083-435a-a451-0a878153d4ec None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:15.269549 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cbb1e-1083-435a-a451-0a878153d4ec None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:15.269994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cbb1e-1083-435a-a451-0a878153d4ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:15.285099 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208cbb1e-1083-435a-a451-0a878153d4ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:45:15.291071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-208cbb1e-1083-435a-a451-0a878153d4ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:45:15.291071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:15.291071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:15.291071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:15.291071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:15.291071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:15.291071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:15.291071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:15.291071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:15.291071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:15.291071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:15.291071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:15.291071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:45:15.291071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:45:15.291071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:45:15.291071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:45:15.292309 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:45:15.292309 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:45:15.292309 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:45:15.292309 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:45:15.292309 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:15.292309 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:15.292309 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:15.292309 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:15.292309 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:15.292309 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:15.292309 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:15.292309 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:15.292309 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:15.292309 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:15.292309 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:15.292309 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:15.293472 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:15.293472 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:15.293472 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:15.293472 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:15.293472 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:45:15.293472 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:45:15.293472 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:45:15.293472 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:45:15.293472 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:45:15.293472 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:45:15.293472 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:45:15.293472 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:15.296051 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 366/737] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:45:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:15.304107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26ea39-37e0-42f4-aa79-4e8690e4bcf5 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:15.304107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26ea39-37e0-42f4-aa79-4e8690e4bcf5 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:15.304107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26ea39-37e0-42f4-aa79-4e8690e4bcf5 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:15.304107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 367/738] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:15.304107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26ea39-37e0-42f4-aa79-4e8690e4bcf5 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:15.379432 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:15.423552 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f5395-5c9f-4a48-a155-c65ea86eab20 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:15.431553 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26ea39-37e0-42f4-aa79-4e8690e4bcf5 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:15.439662 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:15.469378 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654f45c-1c64-4339-870e-d35183b120ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:15.470077 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654f45c-1c64-4339-870e-d35183b120ec None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:15.470721 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654f45c-1c64-4339-870e-d35183b120ec None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:15.471274 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654f45c-1c64-4339-870e-d35183b120ec None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:15.472349 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1654f45c-1c64-4339-870e-d35183b120ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:15.472955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1654f45c-1c64-4339-870e-d35183b120ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:15.474052 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1654f45c-1c64-4339-870e-d35183b120ec None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:15.480175 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddfefbc-eb29-483a-9149-9e38071989c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:15.480941 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfefbc-eb29-483a-9149-9e38071989c6 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:15.481209 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfefbc-eb29-483a-9149-9e38071989c6 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:15.482247 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfefbc-eb29-483a-9149-9e38071989c6 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:15.484020 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddfefbc-eb29-483a-9149-9e38071989c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:15.484652 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddfefbc-eb29-483a-9149-9e38071989c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:15.485898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddfefbc-eb29-483a-9149-9e38071989c6 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:15.487052 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 372/739] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:15.488360 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effef7e8-b345-4896-bb3e-b036cb312bbb None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:15.488642 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effef7e8-b345-4896-bb3e-b036cb312bbb None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:15.488911 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effef7e8-b345-4896-bb3e-b036cb312bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:15.500143 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 368/740] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:45:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:15.501289 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddfefbc-eb29-483a-9149-9e38071989c6 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:15.508798 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-effef7e8-b345-4896-bb3e-b036cb312bbb None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:15.509414 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-effef7e8-b345-4896-bb3e-b036cb312bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:15.597316 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 373/741] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:15 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:45:15.597892 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effef7e8-b345-4896-bb3e-b036cb312bbb None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:15.633858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddfefbc-eb29-483a-9149-9e38071989c6 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:15.666696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:15.678009 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a0bac-b68a-406a-9667-d5bd3bbb5c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:15.678677 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a0bac-b68a-406a-9667-d5bd3bbb5c94 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:15.678951 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a0bac-b68a-406a-9667-d5bd3bbb5c94 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:15.679221 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a0bac-b68a-406a-9667-d5bd3bbb5c94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:15.694171 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a0bac-b68a-406a-9667-d5bd3bbb5c94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:45:15.701708 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-b07a0bac-b68a-406a-9667-d5bd3bbb5c94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:45:15.701708 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:15.701708 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:15.701708 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:15.701708 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:15.701708 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:15.701708 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:15.701708 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:15.701708 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:15.701708 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:15.701708 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:15.701708 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:15.701708 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:45:15.701708 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:45:15.701708 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:45:15.701708 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:45:15.702691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:45:15.702691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:45:15.702691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:45:15.702691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:45:15.702691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:15.702691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:15.702691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:15.702691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:15.702691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:15.702691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:15.702691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:15.702691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:15.702691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:15.702691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:15.702691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:15.702691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:15.709235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:15.709235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:15.709235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:15.709235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:15.709235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:45:15.709235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:45:15.709235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:45:15.709235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:45:15.709235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:45:15.709235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:45:15.709235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:45:15.709235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:15.709235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 369/742] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 31 10:45:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:15.709235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a0bac-b68a-406a-9667-d5bd3bbb5c94 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:15.733367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effef7e8-b345-4896-bb3e-b036cb312bbb None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:15.735572 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:15.775711 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a0414-1912-4971-a8ce-d0bd6a2cde18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:15.777012 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a0414-1912-4971-a8ce-d0bd6a2cde18 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:15.777682 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a0414-1912-4971-a8ce-d0bd6a2cde18 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:15.778362 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a0414-1912-4971-a8ce-d0bd6a2cde18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:15.786568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6a0414-1912-4971-a8ce-d0bd6a2cde18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:15.788475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6a0414-1912-4971-a8ce-d0bd6a2cde18 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:15.803418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-9a6a0414-1912-4971-a8ce-d0bd6a2cde18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:15.803418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:15.803418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:15.803418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:15.803418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:15.803418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:15.803418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:15.803418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:15.803418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:15.803418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:15.803418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:15.803418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:15.803418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:45:15.803418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:45:15.803418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:45:15.804368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:45:15.804368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:15.804368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:15.804368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:15.804368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:15.804368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:15.804368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:15.804368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:15.804368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:15.804368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:15.804368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:15.804368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:15.804368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:15.804368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:15.804368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:15.804368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:15.804368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:15.805245 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:45:15.805245 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:45:15.805245 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:45:15.805245 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:45:15.805245 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:45:15.805245 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:45:15.805245 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:15.805245 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:15.813439 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 374/743] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:45:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:15.814836 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a0414-1912-4971-a8ce-d0bd6a2cde18 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:15.826549 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a0bac-b68a-406a-9667-d5bd3bbb5c94 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:15.871846 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f17c2e-18c1-4936-ad46-62a1cbce49b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:15.872685 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f17c2e-18c1-4936-ad46-62a1cbce49b4 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:15.872685 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f17c2e-18c1-4936-ad46-62a1cbce49b4 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:15.873040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f17c2e-18c1-4936-ad46-62a1cbce49b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:15.885813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f17c2e-18c1-4936-ad46-62a1cbce49b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:15.887990 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f17c2e-18c1-4936-ad46-62a1cbce49b4 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:15.904634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-82f17c2e-18c1-4936-ad46-62a1cbce49b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:15.904634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:15.904634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:15.904634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:15.904634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:15.904634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:15.904634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:15.904634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:15.904634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:15.904634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:15.904634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:15.904634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:15.904634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:15.904634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:15.904634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:15.904634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:15.905626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:15.905626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:15.905626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:15.905626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:15.905626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:15.905626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:15.905626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:15.905626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:15.905626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:15.905626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:15.905626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:15.905626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:15.905626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:15.905626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:15.905626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:15.905626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:15.905626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:15.906548 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:15.906548 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:15.906548 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:15.906548 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 370/744] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:15.906548 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f17c2e-18c1-4936-ad46-62a1cbce49b4 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:15.936808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a0414-1912-4971-a8ce-d0bd6a2cde18 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:15.948821 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:15.976517 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93af884-1758-458f-bd52-ae3cdd47f77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:15.977535 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93af884-1758-458f-bd52-ae3cdd47f77c None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:15.978001 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93af884-1758-458f-bd52-ae3cdd47f77c None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:15.978521 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93af884-1758-458f-bd52-ae3cdd47f77c None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:15.979368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93af884-1758-458f-bd52-ae3cdd47f77c None admin] RBAC: Adding query filter params (name=service) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:15.979870 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93af884-1758-458f-bd52-ae3cdd47f77c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:15.982201 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93af884-1758-458f-bd52-ae3cdd47f77c None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:15.998177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 375/745] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 31 10:45:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:15.999399 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93af884-1758-458f-bd52-ae3cdd47f77c None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:16.013119 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f17c2e-18c1-4936-ad46-62a1cbce49b4 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:16.026999 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:16.052635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cafc1-9551-4fdb-8628-547f5c26042d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:16.053262 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cafc1-9551-4fdb-8628-547f5c26042d None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:16.053540 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cafc1-9551-4fdb-8628-547f5c26042d None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:16.053811 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cafc1-9551-4fdb-8628-547f5c26042d None admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:16.054553 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8cafc1-9551-4fdb-8628-547f5c26042d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:16.054848 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8cafc1-9551-4fdb-8628-547f5c26042d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:16.056363 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8cafc1-9551-4fdb-8628-547f5c26042d None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:16.077024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 371/746] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:45:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:16.077024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cafc1-9551-4fdb-8628-547f5c26042d None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:16.117190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93af884-1758-458f-bd52-ae3cdd47f77c None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:16.153135 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b002b-5fae-4bea-bf7e-eeb2cdf0d830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:16.153761 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b002b-5fae-4bea-bf7e-eeb2cdf0d830 None admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:16.154038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b002b-5fae-4bea-bf7e-eeb2cdf0d830 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:16.154303 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b002b-5fae-4bea-bf7e-eeb2cdf0d830 None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:16.154985 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b002b-5fae-4bea-bf7e-eeb2cdf0d830 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:16.156351 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b002b-5fae-4bea-bf7e-eeb2cdf0d830 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:16.188406 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:16.192164 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cafc1-9551-4fdb-8628-547f5c26042d None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:16.225605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ae929-79b6-40b5-a63a-2bc0653209c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:16.226212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ae929-79b6-40b5-a63a-2bc0653209c3 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:16.226519 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ae929-79b6-40b5-a63a-2bc0653209c3 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:16.226793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ae929-79b6-40b5-a63a-2bc0653209c3 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:16.227430 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596ae929-79b6-40b5-a63a-2bc0653209c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:16.227732 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596ae929-79b6-40b5-a63a-2bc0653209c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:16.228921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 376/747] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Aug 31 10:45:15 2022] POST /identity/v3/services => generated 240 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:45:16.229369 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596ae929-79b6-40b5-a63a-2bc0653209c3 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:16.235531 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b002b-5fae-4bea-bf7e-eeb2cdf0d830 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:16.247813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 372/748] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:16.247813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ae929-79b6-40b5-a63a-2bc0653209c3 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:16.365320 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:16.386082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ae929-79b6-40b5-a63a-2bc0653209c3 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:16.394472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b002b-5fae-4bea-bf7e-eeb2cdf0d830 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:16.432077 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759e8bce-d011-497b-8284-6923d56ec3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:16.442168 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e8bce-d011-497b-8284-6923d56ec3e4 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:16.442168 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e8bce-d011-497b-8284-6923d56ec3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:16.442168 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e8bce-d011-497b-8284-6923d56ec3e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:16.442168 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759e8bce-d011-497b-8284-6923d56ec3e4 None admin] RBAC: Adding query filter params (role.id=13ca33d694c645d6920930d39dc52a0d, scope.project.id=8db9b3f8a9f041938ac7b39003a35015, user.id=e8abc7c439a04622896b34ed0a7b8492) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:16.442168 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759e8bce-d011-497b-8284-6923d56ec3e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:16.442168 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759e8bce-d011-497b-8284-6923d56ec3e4 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:16.442641 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8e8bf-dae3-4db5-9311-71637348f80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:16.442641 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8e8bf-dae3-4db5-9311-71637348f80e None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:16.442641 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8e8bf-dae3-4db5-9311-71637348f80e None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:16.442641 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8e8bf-dae3-4db5-9311-71637348f80e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:16.442909 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 373/749] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:45:16 2022] GET /identity/v3/role_assignments?user.id=e8abc7c439a04622896b34ed0a7b8492&scope.project.id=8db9b3f8a9f041938ac7b39003a35015&role.id=13ca33d694c645d6920930d39dc52a0d => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:16.443217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759e8bce-d011-497b-8284-6923d56ec3e4 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:16.455864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8e8bf-dae3-4db5-9311-71637348f80e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:16.457719 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8e8bf-dae3-4db5-9311-71637348f80e None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:16.477845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-c9d8e8bf-dae3-4db5-9311-71637348f80e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:16.477845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:16.477845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:16.477845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:16.477845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:16.477845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:16.477845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:16.477845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:16.477845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:16.477845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:16.477845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:16.477845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:16.477845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:16.477845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:16.477845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:16.477845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:16.478872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:16.478872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:16.478872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:16.478872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:16.478872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:16.478872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:16.478872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:16.478872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:16.478872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:16.478872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:16.478872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:16.478872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:16.478872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:16.478872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:16.478872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:16.478872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:16.478872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:16.479735 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:16.479735 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:16.479735 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:16.481716 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:16.483366 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 377/750] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:16.489616 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8e8bf-dae3-4db5-9311-71637348f80e None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:16.583127 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759e8bce-d011-497b-8284-6923d56ec3e4 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:16.634114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1530e61b-0ecd-43b4-b6a4-5f77749ccc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:16.635263 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530e61b-0ecd-43b4-b6a4-5f77749ccc89 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:16.635843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530e61b-0ecd-43b4-b6a4-5f77749ccc89 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:16.636287 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530e61b-0ecd-43b4-b6a4-5f77749ccc89 None admin] PATH_INFO: `/v3/users/nova` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:16.652880 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1530e61b-0ecd-43b4-b6a4-5f77749ccc89 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:16.655464 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1530e61b-0ecd-43b4-b6a4-5f77749ccc89 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:16.673388 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-1530e61b-0ecd-43b4-b6a4-5f77749ccc89 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:45:16.673388 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:16.673388 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:16.673388 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:16.673388 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:16.673388 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:16.673388 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:16.673388 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:16.673388 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:16.673388 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:16.673388 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:16.673388 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:16.673388 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:45:16.673388 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:45:16.673388 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:45:16.673388 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:45:16.674366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:16.674366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:16.674366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:45:16.674366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:16.674366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:45:16.674366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:16.674366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:16.674366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:16.674366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:16.674366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:16.674366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:16.674366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:16.674366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:16.674366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:16.674366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:16.674366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:16.674366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:16.674366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:16.675341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:16.675341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:16.675341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:45:16.675341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:45:16.675341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:45:16.675341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:45:16.675341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:45:16.675341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:45:16.675341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:45:16.675341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:16.675341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 374/751] 10.222.0.16 () {56 vars in 1093 bytes} [Wed Aug 31 10:45:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:16.680695 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1530e61b-0ecd-43b4-b6a4-5f77749ccc89 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:16.682379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8e8bf-dae3-4db5-9311-71637348f80e None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:16.722329 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:16.728331 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80be7a7-c447-4834-81e8-f967037a0df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:16.729215 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80be7a7-c447-4834-81e8-f967037a0df9 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:16.739111 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80be7a7-c447-4834-81e8-f967037a0df9 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:16.739111 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80be7a7-c447-4834-81e8-f967037a0df9 None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:16.739111 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80be7a7-c447-4834-81e8-f967037a0df9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:16.739111 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80be7a7-c447-4834-81e8-f967037a0df9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:16.739111 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80be7a7-c447-4834-81e8-f967037a0df9 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:16.750008 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 378/752] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:16.755758 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80be7a7-c447-4834-81e8-f967037a0df9 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:16.790893 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:16.812796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1530e61b-0ecd-43b4-b6a4-5f77749ccc89 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:16.850216 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6f550-d3c1-43a8-9db5-7ee8a353b66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:16.850878 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6f550-d3c1-43a8-9db5-7ee8a353b66c None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:16.851155 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6f550-d3c1-43a8-9db5-7ee8a353b66c None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:16.851426 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6f550-d3c1-43a8-9db5-7ee8a353b66c None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:16.852161 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6f550-d3c1-43a8-9db5-7ee8a353b66c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:16.852466 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6f550-d3c1-43a8-9db5-7ee8a353b66c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:16.854074 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6f550-d3c1-43a8-9db5-7ee8a353b66c None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:16.874846 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 375/753] 10.222.0.16 () {56 vars in 1133 bytes} [Wed Aug 31 10:45:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:16.876127 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b1b9c-55a7-4a54-a687-09785e1c2907 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:16.876410 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b1b9c-55a7-4a54-a687-09785e1c2907 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:16.876686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b1b9c-55a7-4a54-a687-09785e1c2907 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:16.882643 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 376/754] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:16.883448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b1b9c-55a7-4a54-a687-09785e1c2907 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:16.883687 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80be7a7-c447-4834-81e8-f967037a0df9 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:16.921716 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c351db1-ac4b-4cf9-aab6-e4a7e1299377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:16.922353 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c351db1-ac4b-4cf9-aab6-e4a7e1299377 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:16.922667 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c351db1-ac4b-4cf9-aab6-e4a7e1299377 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:16.922953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c351db1-ac4b-4cf9-aab6-e4a7e1299377 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:16.931387 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c351db1-ac4b-4cf9-aab6-e4a7e1299377 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:16.934238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c351db1-ac4b-4cf9-aab6-e4a7e1299377 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:16.945594 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-0c351db1-ac4b-4cf9-aab6-e4a7e1299377 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 10:45:16.945594 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:16.945594 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:16.945594 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:16.945594 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:16.945594 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:16.945594 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:16.945594 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:16.945594 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:16.945594 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:16.945594 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:16.945594 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:16.945594 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:45:16.945594 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:45:16.945594 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:45:16.945594 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:45:16.946832 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:16.946832 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:16.946832 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:45:16.946832 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:16.946832 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:45:16.946832 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:16.946832 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:16.946832 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:16.946832 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:16.946832 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:16.946832 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:16.946832 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:16.946832 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:16.946832 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:16.946832 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:16.946832 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:16.946832 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:16.946832 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:16.947998 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:16.947998 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:16.947998 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:45:16.947998 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:45:16.947998 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:45:16.947998 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:45:16.947998 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:45:16.947998 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:45:16.947998 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 10:45:16.947998 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:16.954523 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 379/755] 10.222.0.16 () {56 vars in 1117 bytes} [Wed Aug 31 10:45:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:16.955959 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13fbd1-e7a6-4f68-8138-c41b7701c27f None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:16.956346 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13fbd1-e7a6-4f68-8138-c41b7701c27f None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:16.956708 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13fbd1-e7a6-4f68-8138-c41b7701c27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:16.977687 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-5e13fbd1-e7a6-4f68-8138-c41b7701c27f None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:16.979963 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-5e13fbd1-e7a6-4f68-8138-c41b7701c27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:17.019601 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b1b9c-55a7-4a54-a687-09785e1c2907 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:17.043989 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:17.060497 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba22aa-6543-4cf3-b202-c06c8bb9185a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:17.061605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba22aa-6543-4cf3-b202-c06c8bb9185a None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:17.064816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba22aa-6543-4cf3-b202-c06c8bb9185a None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:17.064816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba22aa-6543-4cf3-b202-c06c8bb9185a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:17.078095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:17.078511 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 380/756] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:45:17.079439 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13fbd1-e7a6-4f68-8138-c41b7701c27f None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:17.081082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ba22aa-6543-4cf3-b202-c06c8bb9185a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:17.082632 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ba22aa-6543-4cf3-b202-c06c8bb9185a None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:17.110871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-51ba22aa-6543-4cf3-b202-c06c8bb9185a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:17.110871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:17.110871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:17.110871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:17.110871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:17.110871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:17.110871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:17.110871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:17.110871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:17.110871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:17.110871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:17.110871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:17.110871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:17.110871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:17.110871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:17.110871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:17.114418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:17.114418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:17.114418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:17.114418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:17.114418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:17.114418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:17.114418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:17.114418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:17.114418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:17.114418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:17.114418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:17.114418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:17.114418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:17.114418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:17.114418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:17.114418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:17.114418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:17.115307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:17.115307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:17.115307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:17.115307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 377/757] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:17.117926 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba22aa-6543-4cf3-b202-c06c8bb9185a None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:17.281328 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13fbd1-e7a6-4f68-8138-c41b7701c27f None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:17.299227 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba22aa-6543-4cf3-b202-c06c8bb9185a None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:17.342323 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384f9f4-4ed3-4c57-9b2f-1fec359f7abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:17.342995 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384f9f4-4ed3-4c57-9b2f-1fec359f7abb None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:17.343270 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384f9f4-4ed3-4c57-9b2f-1fec359f7abb None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:17.343538 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384f9f4-4ed3-4c57-9b2f-1fec359f7abb None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:17.344174 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384f9f4-4ed3-4c57-9b2f-1fec359f7abb None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:17.344469 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384f9f4-4ed3-4c57-9b2f-1fec359f7abb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:17.345818 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384f9f4-4ed3-4c57-9b2f-1fec359f7abb None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:17.362250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac64bcf-c039-4a02-bbcf-7457313ea7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:17.362912 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac64bcf-c039-4a02-bbcf-7457313ea7b4 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:17.363195 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac64bcf-c039-4a02-bbcf-7457313ea7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:17.363470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac64bcf-c039-4a02-bbcf-7457313ea7b4 None admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:17.364211 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac64bcf-c039-4a02-bbcf-7457313ea7b4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:17.364466 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 378/758] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:17.364830 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac64bcf-c039-4a02-bbcf-7457313ea7b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:17.366274 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac64bcf-c039-4a02-bbcf-7457313ea7b4 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:17.371328 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384f9f4-4ed3-4c57-9b2f-1fec359f7abb None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:17.390539 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 381/759] 10.222.0.16 () {56 vars in 1149 bytes} [Wed Aug 31 10:45:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:17.391564 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac64bcf-c039-4a02-bbcf-7457313ea7b4 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:17.433811 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:17.439194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:17.484319 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384f9f4-4ed3-4c57-9b2f-1fec359f7abb None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:17.502526 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac64bcf-c039-4a02-bbcf-7457313ea7b4 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:17.521736 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b797baa-5b22-4a7f-972f-4f0302ff6cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:17.522510 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b797baa-5b22-4a7f-972f-4f0302ff6cfc None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:17.522785 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b797baa-5b22-4a7f-972f-4f0302ff6cfc None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:17.523052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b797baa-5b22-4a7f-972f-4f0302ff6cfc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:17.523628 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b797baa-5b22-4a7f-972f-4f0302ff6cfc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:17.525080 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b797baa-5b22-4a7f-972f-4f0302ff6cfc None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:17.534634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-4b797baa-5b22-4a7f-972f-4f0302ff6cfc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 10:45:17.534634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:17.534634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:45:17.534634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:45:17.534634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:45:17.534634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:45:17.534634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:45:17.534634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:17.534634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 10:45:17.534634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:17.534634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:45:17.534634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:17.534634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:17.534634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:17.534634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:17.534634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:17.534634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:17.535597 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:17.535597 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:17.535597 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:17.535597 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:17.535597 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:17.535597 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:17.535597 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:45:17.535597 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:45:17.535597 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:45:17.535597 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:45:17.535597 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:17.535597 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:17.535597 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:17.535597 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:17.535597 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:17.535597 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:17.536822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:17.536822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:17.536822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:17.536822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:17.536822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:17.536822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:17.536822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:17.536822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:17.536822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:17.536822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:17.536822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:45:17.536822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:17.536822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 10:45:17.536822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:17.536822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 379/760] 10.222.0.16 () {56 vars in 1129 bytes} [Wed Aug 31 10:45:17 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:17.537590 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b797baa-5b22-4a7f-972f-4f0302ff6cfc None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:17.540166 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c203bb-0a1e-444a-89a6-fa2a45bd0303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:17.540812 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c203bb-0a1e-444a-89a6-fa2a45bd0303 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:17.541146 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c203bb-0a1e-444a-89a6-fa2a45bd0303 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:17.541419 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c203bb-0a1e-444a-89a6-fa2a45bd0303 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:17.550263 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c203bb-0a1e-444a-89a6-fa2a45bd0303 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:17.551785 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c203bb-0a1e-444a-89a6-fa2a45bd0303 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:17.564507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-d0c203bb-0a1e-444a-89a6-fa2a45bd0303 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:17.564507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:17.564507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:17.564507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:17.564507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:17.564507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:17.564507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:17.564507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:17.564507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:17.564507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:17.564507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:17.564507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:17.564507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:45:17.564507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:45:17.564507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:45:17.566683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:45:17.566683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:17.566683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:17.566683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:17.566683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:17.566683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:17.566683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:17.566683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:17.566683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:17.566683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:17.566683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:17.566683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:17.566683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:17.566683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:17.566683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:17.566683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:17.566683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:17.568798 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:45:17.568798 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:45:17.568798 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:45:17.568798 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:45:17.568798 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:45:17.568798 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:45:17.568798 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:17.568798 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:17.568798 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 382/761] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:45:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:17.571745 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c203bb-0a1e-444a-89a6-fa2a45bd0303 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:17.651415 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b797baa-5b22-4a7f-972f-4f0302ff6cfc None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:17.670140 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:17.679849 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:17.683037 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c203bb-0a1e-444a-89a6-fa2a45bd0303 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:17.691100 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465b5a2-62aa-4a14-ba21-cca70e711d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:17.691725 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465b5a2-62aa-4a14-ba21-cca70e711d31 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:17.692002 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465b5a2-62aa-4a14-ba21-cca70e711d31 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:17.692273 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465b5a2-62aa-4a14-ba21-cca70e711d31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:17.722528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa362bb-c2e5-4a72-a09e-87899560b45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:17.723149 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa362bb-c2e5-4a72-a09e-87899560b45d None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:17.723458 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa362bb-c2e5-4a72-a09e-87899560b45d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:17.723738 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa362bb-c2e5-4a72-a09e-87899560b45d None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:17.724433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa362bb-c2e5-4a72-a09e-87899560b45d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:17.724843 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa362bb-c2e5-4a72-a09e-87899560b45d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:17.726252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa362bb-c2e5-4a72-a09e-87899560b45d None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:17.732020 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 383/762] 10.222.0.16 () {56 vars in 1124 bytes} [Wed Aug 31 10:45:17 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:17.732857 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa362bb-c2e5-4a72-a09e-87899560b45d None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:17.766228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b465b5a2-62aa-4a14-ba21-cca70e711d31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:17.767696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b465b5a2-62aa-4a14-ba21-cca70e711d31 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:17.849787 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa362bb-c2e5-4a72-a09e-87899560b45d None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:17.884295 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4933158-e2c0-4997-9404-f9a4ba9e6940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:17.884893 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4933158-e2c0-4997-9404-f9a4ba9e6940 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:17.885165 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4933158-e2c0-4997-9404-f9a4ba9e6940 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:17.885487 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4933158-e2c0-4997-9404-f9a4ba9e6940 None admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:17.886178 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4933158-e2c0-4997-9404-f9a4ba9e6940 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:17.886525 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4933158-e2c0-4997-9404-f9a4ba9e6940 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:17.888099 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4933158-e2c0-4997-9404-f9a4ba9e6940 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:17.893763 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 380/763] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:45:17 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:45:17.894644 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465b5a2-62aa-4a14-ba21-cca70e711d31 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:17.906254 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 384/764] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:45:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:17.907164 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4933158-e2c0-4997-9404-f9a4ba9e6940 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:17.925490 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:17.961378 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:18.031873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4933158-e2c0-4997-9404-f9a4ba9e6940 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:18.032512 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465b5a2-62aa-4a14-ba21-cca70e711d31 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:18.079543 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98733239-27a4-4716-bdf1-b5c82299038a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:18.080561 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98733239-27a4-4716-bdf1-b5c82299038a None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:18.080945 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98733239-27a4-4716-bdf1-b5c82299038a None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:18.081318 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98733239-27a4-4716-bdf1-b5c82299038a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:18.082593 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff51b2-c691-4fe9-8229-68bf8986bff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:18.083213 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff51b2-c691-4fe9-8229-68bf8986bff8 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:18.083493 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff51b2-c691-4fe9-8229-68bf8986bff8 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:18.084086 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff51b2-c691-4fe9-8229-68bf8986bff8 None admin] PATH_INFO: `/v3/services` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:18.085190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff51b2-c691-4fe9-8229-68bf8986bff8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:18.085513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff51b2-c691-4fe9-8229-68bf8986bff8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:18.087000 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff51b2-c691-4fe9-8229-68bf8986bff8 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:18.098425 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 381/765] 10.222.0.16 () {56 vars in 1124 bytes} [Wed Aug 31 10:45:17 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:18.099264 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff51b2-c691-4fe9-8229-68bf8986bff8 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:18.102865 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98733239-27a4-4716-bdf1-b5c82299038a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:18.104635 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98733239-27a4-4716-bdf1-b5c82299038a None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:18.133617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-98733239-27a4-4716-bdf1-b5c82299038a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:18.133617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:18.133617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:18.133617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:18.133617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:18.133617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:18.133617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:18.133617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:18.133617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:18.133617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:18.133617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:18.133617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:18.133617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:18.133617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:18.133617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:18.133617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:18.134665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:18.134665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:18.134665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:18.134665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:18.134665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:18.134665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:18.134665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:18.134665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:18.134665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:18.134665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:18.134665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:18.134665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:18.134665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:18.134665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:18.134665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:18.134665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:18.134665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:18.135677 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:18.135677 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:18.135677 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:18.143210 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 385/766] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:18.143870 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98733239-27a4-4716-bdf1-b5c82299038a None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:18.227731 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:18.257118 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff51b2-c691-4fe9-8229-68bf8986bff8 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:18.266802 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:18.296412 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98733239-27a4-4716-bdf1-b5c82299038a None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:18.312535 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce577b6-19cf-4b21-afaf-57ee7d0a5f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:18.313481 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce577b6-19cf-4b21-afaf-57ee7d0a5f62 None admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:18.313770 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce577b6-19cf-4b21-afaf-57ee7d0a5f62 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:18.314058 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce577b6-19cf-4b21-afaf-57ee7d0a5f62 None admin] PATH_INFO: `/v3/projects/8db9b3f8a9f041938ac7b39003a35015/users/f11971a594014d86968c3356d11d42f7/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:18.316694 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce577b6-19cf-4b21-afaf-57ee7d0a5f62 None admin] RBAC: Authorizing `identity:create_grant(project_id=8db9b3f8a9f041938ac7b39003a35015, user_id=f11971a594014d86968c3356d11d42f7, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:18.318553 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce577b6-19cf-4b21-afaf-57ee7d0a5f62 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:18.342920 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63bc677-6ae8-43b3-8a19-975b5275b35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:18.343663 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63bc677-6ae8-43b3-8a19-975b5275b35a None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:18.344012 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63bc677-6ae8-43b3-8a19-975b5275b35a None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:18.344358 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63bc677-6ae8-43b3-8a19-975b5275b35a None admin] PATH_INFO: `/v3/endpoints` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:18.345134 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63bc677-6ae8-43b3-8a19-975b5275b35a None admin] RBAC: Adding query filter params (interface=public, service_id=29468e8d1fd64ab3bee9044bcce739da, region_id=RegionOne) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:18.345512 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63bc677-6ae8-43b3-8a19-975b5275b35a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:18.347000 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63bc677-6ae8-43b3-8a19-975b5275b35a None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:18.355171 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 386/767] 10.222.0.16 () {56 vars in 1251 bytes} [Wed Aug 31 10:45:18 2022] GET /identity/v3/endpoints?service_id=29468e8d1fd64ab3bee9044bcce739da&interface=public®ion_id=RegionOne => generated 194 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:18.358929 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63bc677-6ae8-43b3-8a19-975b5275b35a None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:18.413918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 382/768] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 31 10:45:18 2022] PUT /identity/v3/projects/8db9b3f8a9f041938ac7b39003a35015/users/f11971a594014d86968c3356d11d42f7/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:45:18.415180 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce577b6-19cf-4b21-afaf-57ee7d0a5f62 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:18.529337 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63bc677-6ae8-43b3-8a19-975b5275b35a None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:18.582364 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a5c58-f55f-419a-9036-3b8ed8f7896c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:18.583216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a5c58-f55f-419a-9036-3b8ed8f7896c None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:18.583594 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a5c58-f55f-419a-9036-3b8ed8f7896c None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:18.583947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a5c58-f55f-419a-9036-3b8ed8f7896c None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:18.584092 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce577b6-19cf-4b21-afaf-57ee7d0a5f62 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:18.584600 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a5c58-f55f-419a-9036-3b8ed8f7896c None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:18.584899 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a5c58-f55f-419a-9036-3b8ed8f7896c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:18.587962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a5c58-f55f-419a-9036-3b8ed8f7896c None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:18.601092 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:18.604975 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 387/769] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:18.613995 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e6e15-0584-480d-9824-991b224864f0 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:18.613995 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e6e15-0584-480d-9824-991b224864f0 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:18.613995 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e6e15-0584-480d-9824-991b224864f0 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:18.613995 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 388/770] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:18.613995 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edc0d8-0c9d-465f-a114-49b1242c4caa None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:18.613995 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edc0d8-0c9d-465f-a114-49b1242c4caa None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:18.614526 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edc0d8-0c9d-465f-a114-49b1242c4caa None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:18.614526 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 389/771] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:18.614526 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1edc0d8-0c9d-465f-a114-49b1242c4caa None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:18.629512 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf57420-fc1f-4f32-9bee-70bd89a2e1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:18.630300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf57420-fc1f-4f32-9bee-70bd89a2e1c3 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:18.631034 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf57420-fc1f-4f32-9bee-70bd89a2e1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:18.631314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf57420-fc1f-4f32-9bee-70bd89a2e1c3 None admin] PATH_INFO: `/v3/services` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:18.631928 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf57420-fc1f-4f32-9bee-70bd89a2e1c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:18.633339 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf57420-fc1f-4f32-9bee-70bd89a2e1c3 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:18.639720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 383/772] 10.222.0.16 () {56 vars in 1087 bytes} [Wed Aug 31 10:45:18 2022] GET /identity/v3/services => generated 772 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:18.641008 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41017c5-6f76-47bb-a26a-9e1c548beaf4 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:18.641294 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41017c5-6f76-47bb-a26a-9e1c548beaf4 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:18.641624 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41017c5-6f76-47bb-a26a-9e1c548beaf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:18.660386 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:18.667520 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-c41017c5-6f76-47bb-a26a-9e1c548beaf4 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:18.671567 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-c41017c5-6f76-47bb-a26a-9e1c548beaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:18.751917 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1edc0d8-0c9d-465f-a114-49b1242c4caa None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:18.770603 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 384/773] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:45:18.772121 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e11b59-f005-4029-89ef-183e38261df2 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:18.772396 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e11b59-f005-4029-89ef-183e38261df2 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:18.772663 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e11b59-f005-4029-89ef-183e38261df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:18.814361 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81579e72-2705-4859-a375-bfa8c8fdbf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:18.815061 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81579e72-2705-4859-a375-bfa8c8fdbf98 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:18.815351 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81579e72-2705-4859-a375-bfa8c8fdbf98 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:18.815622 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81579e72-2705-4859-a375-bfa8c8fdbf98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:18.825432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81579e72-2705-4859-a375-bfa8c8fdbf98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:18.826990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81579e72-2705-4859-a375-bfa8c8fdbf98 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:18.841315 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-81579e72-2705-4859-a375-bfa8c8fdbf98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:18.841315 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:18.841315 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:18.841315 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:18.841315 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:18.841315 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:18.841315 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:18.841315 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:18.841315 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:18.841315 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:18.841315 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:18.841315 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:18.841315 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:45:18.841315 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:45:18.841315 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:45:18.842274 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:45:18.842274 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:18.842274 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:18.842274 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:18.842274 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:18.842274 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:18.842274 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:18.842274 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:18.842274 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:18.842274 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:18.842274 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:18.842274 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:18.842274 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:18.842274 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:18.842274 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:18.842274 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:18.842274 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:18.843170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:45:18.843170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:45:18.843170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:45:18.843170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:45:18.843170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:45:18.843170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:45:18.843170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:18.843170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:18.843170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 390/774] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:45:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:18.850422 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81579e72-2705-4859-a375-bfa8c8fdbf98 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:18.917534 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:18.919208 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-d5e11b59-f005-4029-89ef-183e38261df2 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:18.920651 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-d5e11b59-f005-4029-89ef-183e38261df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:18.989240 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81579e72-2705-4859-a375-bfa8c8fdbf98 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:19.026456 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 385/775] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:45:19.040929 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd37b50-7a72-4116-a77b-762d603fce52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:19.041531 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd37b50-7a72-4116-a77b-762d603fce52 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:19.041805 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd37b50-7a72-4116-a77b-762d603fce52 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:19.042071 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd37b50-7a72-4116-a77b-762d603fce52 None admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:19.042866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd37b50-7a72-4116-a77b-762d603fce52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:19.043167 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd37b50-7a72-4116-a77b-762d603fce52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:19.044717 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd37b50-7a72-4116-a77b-762d603fce52 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:19.058526 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 391/776] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:45:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:19.067577 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e11b59-f005-4029-89ef-183e38261df2 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:19.098076 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd37b50-7a72-4116-a77b-762d603fce52 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:19.184314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:19.243101 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e11b59-f005-4029-89ef-183e38261df2 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:19.289413 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd37b50-7a72-4116-a77b-762d603fce52 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:19.295431 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a26f5-6870-4539-8b0a-983a399b05e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:19.296095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a26f5-6870-4539-8b0a-983a399b05e2 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:19.296369 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a26f5-6870-4539-8b0a-983a399b05e2 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:19.296639 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a26f5-6870-4539-8b0a-983a399b05e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:19.297365 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9a26f5-6870-4539-8b0a-983a399b05e2 None admin] RBAC: Adding query filter params (role.id=bec755d6565a41798b09f5f57c1d51dd, scope.project.id=8db9b3f8a9f041938ac7b39003a35015, user.id=7796ebff70054de090e02c58bf7000f4) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:19.297670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9a26f5-6870-4539-8b0a-983a399b05e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:19.299123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9a26f5-6870-4539-8b0a-983a399b05e2 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:19.305481 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 386/777] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:45:19 2022] GET /identity/v3/role_assignments?user.id=7796ebff70054de090e02c58bf7000f4&scope.project.id=8db9b3f8a9f041938ac7b39003a35015&role.id=bec755d6565a41798b09f5f57c1d51dd => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:19.380004 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:19.385613 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a26f5-6870-4539-8b0a-983a399b05e2 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:19.398266 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab4dd74-d5dd-4758-81b7-b807f81c86bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:19.398925 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4dd74-d5dd-4758-81b7-b807f81c86bf None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:19.399201 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4dd74-d5dd-4758-81b7-b807f81c86bf None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:19.399470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4dd74-d5dd-4758-81b7-b807f81c86bf None admin] PATH_INFO: `/v3/services/network` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:19.400047 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab4dd74-d5dd-4758-81b7-b807f81c86bf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:19.401444 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab4dd74-d5dd-4758-81b7-b807f81c86bf None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:19.410568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-8ab4dd74-d5dd-4758-81b7-b807f81c86bf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 10:45:19.410568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:19.410568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:45:19.410568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:45:19.410568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:45:19.410568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:45:19.410568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:45:19.410568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:19.410568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 10:45:19.410568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:19.410568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:45:19.410568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:19.410568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:19.410568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:19.410568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:19.410568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:19.410568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:19.411634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:19.411634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:19.411634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:19.411634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:19.411634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:19.411634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:19.411634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:45:19.411634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:45:19.411634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:45:19.411634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:45:19.411634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:19.411634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:19.411634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:19.411634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:19.411634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:19.411634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:19.412466 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:19.412466 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:19.412466 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:19.412466 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:19.412466 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:19.412466 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:19.412466 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:19.412466 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:19.412466 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:19.412466 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:19.412466 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:45:19.412466 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:19.412466 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 10:45:19.412466 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:19.412466 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 392/778] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:45:19 2022] GET /identity/v3/services/network => generated 88 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:19.418050 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab4dd74-d5dd-4758-81b7-b807f81c86bf None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:19.540737 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a26f5-6870-4539-8b0a-983a399b05e2 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:19.555260 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab4dd74-d5dd-4758-81b7-b807f81c86bf None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:19.592803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:19.607287 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047737f2-6980-49f0-8cdf-16faf686eb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:19.609027 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047737f2-6980-49f0-8cdf-16faf686eb36 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:19.609305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047737f2-6980-49f0-8cdf-16faf686eb36 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:19.609576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047737f2-6980-49f0-8cdf-16faf686eb36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:19.628951 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f39601-a1a4-40c4-b343-2cd74b3e5d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:19.629562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f39601-a1a4-40c4-b343-2cd74b3e5d4c None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:19.629839 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f39601-a1a4-40c4-b343-2cd74b3e5d4c None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:19.630130 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f39601-a1a4-40c4-b343-2cd74b3e5d4c None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:19.630797 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f39601-a1a4-40c4-b343-2cd74b3e5d4c None admin] RBAC: Adding query filter params (name=network) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:19.631098 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f39601-a1a4-40c4-b343-2cd74b3e5d4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:19.632815 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f39601-a1a4-40c4-b343-2cd74b3e5d4c None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:19.634359 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047737f2-6980-49f0-8cdf-16faf686eb36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:45:19.640356 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-047737f2-6980-49f0-8cdf-16faf686eb36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:45:19.640356 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:19.640356 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:19.640356 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:19.640356 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:19.640356 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:19.640356 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:19.640356 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:19.640356 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:19.640356 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:19.640356 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:19.640356 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:19.640356 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:45:19.640356 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:45:19.640356 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:45:19.640356 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:45:19.641300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:45:19.641300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:45:19.641300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:45:19.641300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:45:19.641300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:19.641300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:19.641300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:19.641300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:19.641300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:19.641300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:19.641300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:19.641300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:19.641300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:19.641300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:19.641300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:19.641300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:19.642124 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:19.642124 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:19.642124 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:19.642124 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:19.642124 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:45:19.642124 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:45:19.642124 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:45:19.642124 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:45:19.642124 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:45:19.642124 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:45:19.642124 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:45:19.642124 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:19.642124 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 387/779] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:45:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:19.645002 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 393/780] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Aug 31 10:45:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:19.647693 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047737f2-6980-49f0-8cdf-16faf686eb36 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:19.653617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f39601-a1a4-40c4-b343-2cd74b3e5d4c None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:19.764673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047737f2-6980-49f0-8cdf-16faf686eb36 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:19.804473 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2440cd0-7cf4-417f-8b4c-cda210bb795f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:19.805008 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2440cd0-7cf4-417f-8b4c-cda210bb795f None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:19.805286 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2440cd0-7cf4-417f-8b4c-cda210bb795f None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:19.805560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2440cd0-7cf4-417f-8b4c-cda210bb795f None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:19.806698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2440cd0-7cf4-417f-8b4c-cda210bb795f None admin] RBAC: Adding query filter params (name=admin) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:19.807003 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2440cd0-7cf4-417f-8b4c-cda210bb795f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:19.808432 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2440cd0-7cf4-417f-8b4c-cda210bb795f None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:19.817038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:19.818919 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 388/781] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:45:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:19.824765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2440cd0-7cf4-417f-8b4c-cda210bb795f None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:19.908761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:19.936534 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f39601-a1a4-40c4-b343-2cd74b3e5d4c None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:19.995498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2440cd0-7cf4-417f-8b4c-cda210bb795f None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:20.010275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e15dd-9032-415f-be15-544251dfccbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:20.012643 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e15dd-9032-415f-be15-544251dfccbb None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:20.012643 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e15dd-9032-415f-be15-544251dfccbb None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:20.012643 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e15dd-9032-415f-be15-544251dfccbb None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:20.013131 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e15dd-9032-415f-be15-544251dfccbb None admin] RBAC: Adding query filter params (type=network) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:20.013131 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e15dd-9032-415f-be15-544251dfccbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:20.022646 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e15dd-9032-415f-be15-544251dfccbb None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:20.027922 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 394/782] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Aug 31 10:45:19 2022] GET /identity/v3/services?type=network => generated 350 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:20.035120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e15dd-9032-415f-be15-544251dfccbb None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:20.080253 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4ae5d-b627-48f7-8eb4-9c88efe4ce08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:20.080812 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4ae5d-b627-48f7-8eb4-9c88efe4ce08 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:20.081083 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4ae5d-b627-48f7-8eb4-9c88efe4ce08 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:20.081349 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4ae5d-b627-48f7-8eb4-9c88efe4ce08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:20.106981 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4ae5d-b627-48f7-8eb4-9c88efe4ce08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:20.108489 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4ae5d-b627-48f7-8eb4-9c88efe4ce08 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:20.127331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-f3c4ae5d-b627-48f7-8eb4-9c88efe4ce08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:20.127331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:20.127331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:20.127331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:20.127331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:20.127331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:20.127331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:20.127331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:20.127331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:20.127331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:20.127331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:20.127331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:20.127331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:20.127331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:20.127331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:20.127331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:20.128342 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:20.128342 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:20.128342 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:20.128342 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:20.128342 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:20.128342 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:20.128342 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:20.128342 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:20.128342 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:20.128342 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:20.128342 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:20.128342 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:20.128342 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:20.128342 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:20.128342 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:20.128342 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:20.128342 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:20.129212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:20.129212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:20.129212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:20.131315 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 389/783] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:20.137641 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4ae5d-b627-48f7-8eb4-9c88efe4ce08 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:20.255843 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e15dd-9032-415f-be15-544251dfccbb None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:20.278762 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:20.300309 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4ae5d-b627-48f7-8eb4-9c88efe4ce08 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:20.303093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:20.337690 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e1d95-329f-4280-a521-8f034e070c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:20.338215 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e1d95-329f-4280-a521-8f034e070c39 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:20.338516 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e1d95-329f-4280-a521-8f034e070c39 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:20.338785 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e1d95-329f-4280-a521-8f034e070c39 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:20.339431 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994e1d95-329f-4280-a521-8f034e070c39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:20.339773 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994e1d95-329f-4280-a521-8f034e070c39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:20.340427 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c634a8-02ff-4168-85dc-4b776e38e86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:20.341347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c634a8-02ff-4168-85dc-4b776e38e86f None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:20.341581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994e1d95-329f-4280-a521-8f034e070c39 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:20.342040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c634a8-02ff-4168-85dc-4b776e38e86f None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:20.342551 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c634a8-02ff-4168-85dc-4b776e38e86f None admin] PATH_INFO: `/v3/endpoints` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:20.350420 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c634a8-02ff-4168-85dc-4b776e38e86f None admin] RBAC: Adding query filter params (interface=public, service_id=be7d1d582fe04160b196547667b10c76, region_id=RegionOne) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:20.350936 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c634a8-02ff-4168-85dc-4b776e38e86f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:20.352362 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c634a8-02ff-4168-85dc-4b776e38e86f None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:20.354905 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 390/784] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:20.361086 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e1d95-329f-4280-a521-8f034e070c39 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:20.368605 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 395/785] 10.222.0.16 () {56 vars in 1251 bytes} [Wed Aug 31 10:45:20 2022] GET /identity/v3/endpoints?service_id=be7d1d582fe04160b196547667b10c76&interface=public®ion_id=RegionOne => generated 194 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:20.377353 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c634a8-02ff-4168-85dc-4b776e38e86f None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:20.474698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e1d95-329f-4280-a521-8f034e070c39 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:20.514588 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09dabd-d225-440b-81fd-2ca1decf0757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:20.515339 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09dabd-d225-440b-81fd-2ca1decf0757 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:20.515619 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09dabd-d225-440b-81fd-2ca1decf0757 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:20.515889 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09dabd-d225-440b-81fd-2ca1decf0757 None admin] PATH_INFO: `/v3/users/swift` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:20.529781 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de09dabd-d225-440b-81fd-2ca1decf0757 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:20.531267 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de09dabd-d225-440b-81fd-2ca1decf0757 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:20.553799 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-de09dabd-d225-440b-81fd-2ca1decf0757 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 10:45:20.553799 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:20.553799 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:20.553799 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:20.553799 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:20.553799 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:20.553799 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:20.553799 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:20.553799 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:20.553799 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:20.553799 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:20.553799 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:20.553799 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:45:20.553799 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:45:20.553799 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:45:20.553799 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:45:20.554850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:20.554850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:20.554850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:45:20.554850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:20.554850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:45:20.554850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:20.554850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:20.554850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:20.554850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:20.554850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:20.554850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:20.554850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:20.554850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:20.554850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:20.554850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:20.554850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:20.554850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:20.554850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:20.555773 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:20.555773 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:20.555773 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:45:20.555773 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:45:20.555773 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:45:20.555773 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:45:20.555773 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:45:20.555773 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:45:20.555773 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 10:45:20.555773 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:20.560642 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 391/786] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:45:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:20.568422 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09dabd-d225-440b-81fd-2ca1decf0757 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:20.612561 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c634a8-02ff-4168-85dc-4b776e38e86f None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:20.618801 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:20.700414 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d76ea-d880-4e05-b279-bdd0313f7729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:20.701403 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d76ea-d880-4e05-b279-bdd0313f7729 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:20.701916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d76ea-d880-4e05-b279-bdd0313f7729 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:20.702415 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d76ea-d880-4e05-b279-bdd0313f7729 None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:20.703246 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1d76ea-d880-4e05-b279-bdd0313f7729 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:20.704857 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1d76ea-d880-4e05-b279-bdd0313f7729 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:20.721091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 396/787] 10.222.0.16 () {56 vars in 1087 bytes} [Wed Aug 31 10:45:20 2022] GET /identity/v3/services => generated 772 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:20.742220 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09dabd-d225-440b-81fd-2ca1decf0757 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:20.769190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c9213-4573-4461-b140-4c4fdae0b72e None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:20.769830 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c9213-4573-4461-b140-4c4fdae0b72e None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:20.770313 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c9213-4573-4461-b140-4c4fdae0b72e None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:20.772531 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 397/788] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:20.779068 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583dbaee-8c05-417b-b538-7ed419db89b6 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:20.779701 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583dbaee-8c05-417b-b538-7ed419db89b6 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:20.780205 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583dbaee-8c05-417b-b538-7ed419db89b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:20.784271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daecab2c-594f-4f4d-b955-640acc69456f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:20.784946 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecab2c-594f-4f4d-b955-640acc69456f None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:20.785223 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecab2c-594f-4f4d-b955-640acc69456f None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:20.785496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecab2c-594f-4f4d-b955-640acc69456f None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:20.786299 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daecab2c-594f-4f4d-b955-640acc69456f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:20.786638 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daecab2c-594f-4f4d-b955-640acc69456f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:20.788156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daecab2c-594f-4f4d-b955-640acc69456f None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:20.803355 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-583dbaee-8c05-417b-b538-7ed419db89b6 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:20.805012 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-583dbaee-8c05-417b-b538-7ed419db89b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:20.823383 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 392/789] 10.222.0.16 () {56 vars in 1135 bytes} [Wed Aug 31 10:45:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:20.831960 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daecab2c-594f-4f4d-b955-640acc69456f None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:20.885551 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:20.954206 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 398/790] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:45:20.989671 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:21.011290 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daecab2c-594f-4f4d-b955-640acc69456f None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:21.047947 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c49767-5a57-4e62-afbd-8a1563913a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:21.048671 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c49767-5a57-4e62-afbd-8a1563913a71 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:21.048946 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c49767-5a57-4e62-afbd-8a1563913a71 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:21.049209 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c49767-5a57-4e62-afbd-8a1563913a71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:21.049984 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664e33c-cc53-4919-9ccc-f2294fd3596d None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:21.050511 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664e33c-cc53-4919-9ccc-f2294fd3596d None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:21.051119 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664e33c-cc53-4919-9ccc-f2294fd3596d None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:21.053087 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 399/791] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:21.059416 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531a779-3556-49b2-a449-fb56d552200b None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:21.060037 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531a779-3556-49b2-a449-fb56d552200b None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:21.060512 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531a779-3556-49b2-a449-fb56d552200b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:21.060899 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c49767-5a57-4e62-afbd-8a1563913a71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:21.062351 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c49767-5a57-4e62-afbd-8a1563913a71 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:21.079504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-c0c49767-5a57-4e62-afbd-8a1563913a71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:21.079504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:21.079504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:21.079504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:21.079504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:21.079504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:21.079504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:21.079504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:21.079504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:21.079504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:21.079504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:21.079504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:21.079504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:21.079504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:21.079504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:21.079504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:21.080482 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:21.080482 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:21.080482 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:21.080482 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:21.080482 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:21.080482 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:21.080482 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:21.080482 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:21.080482 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:21.080482 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:21.080482 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:21.080482 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:21.080482 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:21.080482 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:21.080482 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:21.080482 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:21.080482 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:21.081395 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:21.081395 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:21.081395 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:21.081395 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 393/792] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:21.090184 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c49767-5a57-4e62-afbd-8a1563913a71 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:21.100600 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-c531a779-3556-49b2-a449-fb56d552200b None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:21.102244 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-c531a779-3556-49b2-a449-fb56d552200b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:21.220983 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c49767-5a57-4e62-afbd-8a1563913a71 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:21.224961 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 400/793] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:45:21.253550 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:21.260474 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fdd1d-62a5-4b96-9b1f-219840906b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:21.260474 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fdd1d-62a5-4b96-9b1f-219840906b05 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:21.260474 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fdd1d-62a5-4b96-9b1f-219840906b05 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:21.260474 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fdd1d-62a5-4b96-9b1f-219840906b05 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:21.260810 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007fdd1d-62a5-4b96-9b1f-219840906b05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:21.260810 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007fdd1d-62a5-4b96-9b1f-219840906b05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:21.262338 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007fdd1d-62a5-4b96-9b1f-219840906b05 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:21.276099 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 394/794] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:21.281816 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531a779-3556-49b2-a449-fb56d552200b None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:21.299790 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fdd1d-62a5-4b96-9b1f-219840906b05 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:21.445036 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fdd1d-62a5-4b96-9b1f-219840906b05 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:21.462814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:21.472378 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531a779-3556-49b2-a449-fb56d552200b None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:21.529809 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31a257-52fd-475e-a097-712ba5a6605d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:21.530600 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31a257-52fd-475e-a097-712ba5a6605d None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:21.530939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31a257-52fd-475e-a097-712ba5a6605d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:21.531217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31a257-52fd-475e-a097-712ba5a6605d None admin] PATH_INFO: `/v3/projects/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:21.539910 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c31a257-52fd-475e-a097-712ba5a6605d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:21.541569 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c31a257-52fd-475e-a097-712ba5a6605d None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:21.554473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-3c31a257-52fd-475e-a097-712ba5a6605d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:21.554473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:21.554473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:21.554473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:21.554473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:21.554473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:21.554473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:21.554473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:21.554473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:21.554473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:21.554473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:21.554473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:21.554473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:45:21.554473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:45:21.554473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:45:21.556423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:45:21.556423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:21.556423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:21.556423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:21.556423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:21.556423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:21.556423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:21.556423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:21.556423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:21.556423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:21.556423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:21.556423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:21.556423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:21.556423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:21.556423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:21.556423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:21.556423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:21.557818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:45:21.557818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:45:21.557818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:45:21.557818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:45:21.557818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:45:21.557818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:45:21.557818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:21.557818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:21.557818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 401/795] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:45:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:21.557818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31a257-52fd-475e-a097-712ba5a6605d None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:21.559855 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057268b-6e41-4b39-b91f-db6eea23cf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:21.560686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057268b-6e41-4b39-b91f-db6eea23cf9b None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:21.561283 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057268b-6e41-4b39-b91f-db6eea23cf9b None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:21.561939 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057268b-6e41-4b39-b91f-db6eea23cf9b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:21.578223 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057268b-6e41-4b39-b91f-db6eea23cf9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:45:21.583622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-f057268b-6e41-4b39-b91f-db6eea23cf9b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:45:21.583622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:21.583622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:21.583622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:21.583622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:21.583622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:21.583622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:21.583622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:21.583622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:21.583622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:21.583622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:21.583622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:21.583622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:45:21.583622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:45:21.583622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:45:21.583622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:45:21.584557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:45:21.584557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:45:21.584557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:45:21.584557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:45:21.584557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:21.584557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:21.584557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:21.584557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:21.584557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:21.584557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:21.584557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:21.584557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:21.584557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:21.584557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:21.584557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:21.584557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:21.585629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:21.585629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:21.585629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:21.585629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:21.585629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:45:21.585629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:45:21.585629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:45:21.585629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:45:21.585629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:45:21.585629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:45:21.585629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:45:21.585629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:21.585629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 395/796] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:45:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:21.586661 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057268b-6e41-4b39-b91f-db6eea23cf9b None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:21.663433 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:21.690256 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31a257-52fd-475e-a097-712ba5a6605d None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:21.709929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057268b-6e41-4b39-b91f-db6eea23cf9b None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:21.732156 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c845ab22-e2f2-4fe1-870c-a894eab42d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:21.733208 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845ab22-e2f2-4fe1-870c-a894eab42d61 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:21.733495 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845ab22-e2f2-4fe1-870c-a894eab42d61 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:21.733907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845ab22-e2f2-4fe1-870c-a894eab42d61 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:21.734368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c845ab22-e2f2-4fe1-870c-a894eab42d61 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:21.735811 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c845ab22-e2f2-4fe1-870c-a894eab42d61 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:21.748446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-c845ab22-e2f2-4fe1-870c-a894eab42d61 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 10:45:21.748446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:21.748446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:45:21.748446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:45:21.748446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:45:21.748446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:45:21.748446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:45:21.748446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:21.748446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 10:45:21.748446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:21.748446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:45:21.748446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:21.748446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:21.748446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:21.748446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:21.748446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:21.748446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:21.749472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:21.749472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:21.749472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:21.749472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:21.749472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:21.749472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:21.749472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:45:21.749472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:45:21.749472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:45:21.749472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:45:21.749472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:21.749472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:21.749472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:21.749472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:21.749472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:21.749472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:21.750321 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:21.750321 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:21.750321 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:21.750321 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:21.750321 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:21.750321 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:21.750321 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:21.750321 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:21.750321 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:21.750321 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:21.750321 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:45:21.750321 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:21.750321 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 10:45:21.750321 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:21.750321 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 402/797] 10.222.0.16 () {56 vars in 1129 bytes} [Wed Aug 31 10:45:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:21.751135 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c845ab22-e2f2-4fe1-870c-a894eab42d61 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:21.755031 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee6fe2b-5d14-41b8-8d07-2b06c8696084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:21.755664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6fe2b-5d14-41b8-8d07-2b06c8696084 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:21.755947 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6fe2b-5d14-41b8-8d07-2b06c8696084 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:21.756219 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6fe2b-5d14-41b8-8d07-2b06c8696084 None admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:21.756922 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee6fe2b-5d14-41b8-8d07-2b06c8696084 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:21.757220 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee6fe2b-5d14-41b8-8d07-2b06c8696084 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:21.758875 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee6fe2b-5d14-41b8-8d07-2b06c8696084 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:21.771685 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 396/798] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:45:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:21.772898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48763552-39e2-4848-9423-df2eceb9858a None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:21.773186 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48763552-39e2-4848-9423-df2eceb9858a None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:21.773455 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48763552-39e2-4848-9423-df2eceb9858a None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:21.779847 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 397/799] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:21.779847 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48763552-39e2-4848-9423-df2eceb9858a None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:21.792243 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:21.873328 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c845ab22-e2f2-4fe1-870c-a894eab42d61 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:21.895274 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48763552-39e2-4848-9423-df2eceb9858a None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:21.910624 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:21.922292 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc7645-ca20-4db1-a75c-68d845f6f3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:21.923933 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc7645-ca20-4db1-a75c-68d845f6f3e5 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:21.923933 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc7645-ca20-4db1-a75c-68d845f6f3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:21.923933 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc7645-ca20-4db1-a75c-68d845f6f3e5 None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:21.931999 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc7645-ca20-4db1-a75c-68d845f6f3e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:21.931999 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc7645-ca20-4db1-a75c-68d845f6f3e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:21.931999 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc7645-ca20-4db1-a75c-68d845f6f3e5 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:21.935212 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 403/800] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:45:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:21.936219 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b5100-4bc6-45f8-b213-65893901fb06 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:21.938384 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b5100-4bc6-45f8-b213-65893901fb06 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:21.938384 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b5100-4bc6-45f8-b213-65893901fb06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:21.952233 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d94787-a52d-4c27-ad00-b9be40ff5133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:21.952848 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d94787-a52d-4c27-ad00-b9be40ff5133 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:21.953125 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d94787-a52d-4c27-ad00-b9be40ff5133 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:21.953395 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d94787-a52d-4c27-ad00-b9be40ff5133 None admin] PATH_INFO: `/v3/services` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:21.954051 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d94787-a52d-4c27-ad00-b9be40ff5133 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:21.954354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d94787-a52d-4c27-ad00-b9be40ff5133 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:21.955472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-2d4b5100-4bc6-45f8-b213-65893901fb06 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:21.955852 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d94787-a52d-4c27-ad00-b9be40ff5133 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:21.956674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2d4b5100-4bc6-45f8-b213-65893901fb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:21.960881 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 398/801] 10.222.0.16 () {56 vars in 1124 bytes} [Wed Aug 31 10:45:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:21.967063 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d94787-a52d-4c27-ad00-b9be40ff5133 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:22.041381 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 404/802] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:45:22.042531 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4b5100-4bc6-45f8-b213-65893901fb06 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:22.071880 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:22.106418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d94787-a52d-4c27-ad00-b9be40ff5133 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:22.148872 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed613e5-2a60-4e11-9a9a-f0efef94c9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:22.150335 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed613e5-2a60-4e11-9a9a-f0efef94c9e4 None admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:22.150637 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed613e5-2a60-4e11-9a9a-f0efef94c9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:22.150908 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed613e5-2a60-4e11-9a9a-f0efef94c9e4 None admin] PATH_INFO: `/v3/projects/8db9b3f8a9f041938ac7b39003a35015/users/e8abc7c439a04622896b34ed0a7b8492/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:22.153200 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed613e5-2a60-4e11-9a9a-f0efef94c9e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=8db9b3f8a9f041938ac7b39003a35015, user_id=e8abc7c439a04622896b34ed0a7b8492, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:22.155160 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed613e5-2a60-4e11-9a9a-f0efef94c9e4 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:22.186990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4b5100-4bc6-45f8-b213-65893901fb06 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:22.234166 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53772dc-bd61-4c8d-9014-88e0d5b91084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:22.235234 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53772dc-bd61-4c8d-9014-88e0d5b91084 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:22.235723 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53772dc-bd61-4c8d-9014-88e0d5b91084 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:22.236269 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53772dc-bd61-4c8d-9014-88e0d5b91084 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:22.238171 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 399/803] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 31 10:45:21 2022] PUT /identity/v3/projects/8db9b3f8a9f041938ac7b39003a35015/users/e8abc7c439a04622896b34ed0a7b8492/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:45:22.239191 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed613e5-2a60-4e11-9a9a-f0efef94c9e4 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:22.252840 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53772dc-bd61-4c8d-9014-88e0d5b91084 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:22.255316 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53772dc-bd61-4c8d-9014-88e0d5b91084 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:22.278886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-d53772dc-bd61-4c8d-9014-88e0d5b91084 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:22.278886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:22.278886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:22.278886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:22.278886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:22.278886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:22.278886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:22.278886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:22.278886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:22.278886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:22.278886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:22.278886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:22.278886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:22.278886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:22.278886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:22.278886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:22.279960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:22.279960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:22.279960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:22.279960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:22.279960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:22.279960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:22.279960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:22.279960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:22.279960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:22.279960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:22.279960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:22.279960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:22.279960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:22.279960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:22.279960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:22.279960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:22.279960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:22.281982 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:22.281982 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:22.281982 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:22.283954 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 405/804] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:22.289811 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53772dc-bd61-4c8d-9014-88e0d5b91084 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:22.321361 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:22.406017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:22.424614 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed613e5-2a60-4e11-9a9a-f0efef94c9e4 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:22.452532 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53772dc-bd61-4c8d-9014-88e0d5b91084 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:22.473478 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b579ad-e0b3-4fd4-be1c-50ab0bd09d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:22.474374 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b579ad-e0b3-4fd4-be1c-50ab0bd09d6e None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:22.474676 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b579ad-e0b3-4fd4-be1c-50ab0bd09d6e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:22.474954 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b579ad-e0b3-4fd4-be1c-50ab0bd09d6e None admin] PATH_INFO: `/v3/services` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:22.475865 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b579ad-e0b3-4fd4-be1c-50ab0bd09d6e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:22.476567 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b579ad-e0b3-4fd4-be1c-50ab0bd09d6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:22.477956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b579ad-e0b3-4fd4-be1c-50ab0bd09d6e None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:22.485859 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 400/805] 10.222.0.16 () {56 vars in 1124 bytes} [Wed Aug 31 10:45:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:22.486787 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b579ad-e0b3-4fd4-be1c-50ab0bd09d6e None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:22.502133 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5bc99-4027-4ad1-97aa-5bba2fc95087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:22.502774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5bc99-4027-4ad1-97aa-5bba2fc95087 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:22.503050 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5bc99-4027-4ad1-97aa-5bba2fc95087 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:22.503318 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5bc99-4027-4ad1-97aa-5bba2fc95087 None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:22.503959 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e5bc99-4027-4ad1-97aa-5bba2fc95087 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:22.504259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e5bc99-4027-4ad1-97aa-5bba2fc95087 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:22.505627 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e5bc99-4027-4ad1-97aa-5bba2fc95087 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:22.520077 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 406/806] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:22.520911 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5bc99-4027-4ad1-97aa-5bba2fc95087 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:22.621762 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:22.625440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b579ad-e0b3-4fd4-be1c-50ab0bd09d6e None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:22.637881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5bc99-4027-4ad1-97aa-5bba2fc95087 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:22.670123 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:22.672709 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3608441d-4810-4e95-97e1-8e090684fdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:22.673553 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608441d-4810-4e95-97e1-8e090684fdd2 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:22.674128 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608441d-4810-4e95-97e1-8e090684fdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:22.674475 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608441d-4810-4e95-97e1-8e090684fdd2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:22.678600 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eafa4c2-669c-4893-9e9c-928ad4a79fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:22.679307 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafa4c2-669c-4893-9e9c-928ad4a79fba None admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:22.679642 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafa4c2-669c-4893-9e9c-928ad4a79fba None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:22.679971 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafa4c2-669c-4893-9e9c-928ad4a79fba None admin] PATH_INFO: `/v3/endpoints` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:22.680700 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eafa4c2-669c-4893-9e9c-928ad4a79fba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:22.684387 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eafa4c2-669c-4893-9e9c-928ad4a79fba None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:22.686170 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3608441d-4810-4e95-97e1-8e090684fdd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:45:22.691971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-3608441d-4810-4e95-97e1-8e090684fdd2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:45:22.691971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:22.691971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:22.691971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:22.691971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:22.691971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:22.691971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:22.691971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:22.691971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:22.691971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:22.691971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:22.691971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:22.691971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:45:22.691971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:45:22.691971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:45:22.691971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:45:22.693095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:45:22.693095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:45:22.693095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:45:22.693095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:45:22.693095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:22.693095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:22.693095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:22.693095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:22.693095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:22.693095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:22.693095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:22.693095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:22.693095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:22.693095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:22.693095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:22.693095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:22.694304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:22.694304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:22.694304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:22.694304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:22.694304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:45:22.694304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:45:22.694304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:45:22.694304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:45:22.694304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:45:22.694304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:45:22.694304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:45:22.694304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:22.695493 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 401/807] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 31 10:45:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:22.695607 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3608441d-4810-4e95-97e1-8e090684fdd2 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:22.773733 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 407/808] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 10:45:22 2022] POST /identity/v3/endpoints => generated 346 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:45:22.779489 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eafa4c2-669c-4893-9e9c-928ad4a79fba None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:22.814252 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3608441d-4810-4e95-97e1-8e090684fdd2 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:22.867444 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045217a-a261-4377-b3d0-aa607511d7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:22.868271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045217a-a261-4377-b3d0-aa607511d7fa None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:22.868617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045217a-a261-4377-b3d0-aa607511d7fa None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:22.868951 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045217a-a261-4377-b3d0-aa607511d7fa None admin] PATH_INFO: `/v3/users/nova` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:22.878765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5045217a-a261-4377-b3d0-aa607511d7fa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:22.880250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5045217a-a261-4377-b3d0-aa607511d7fa None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:22.895114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-5045217a-a261-4377-b3d0-aa607511d7fa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:45:22.895114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:22.895114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:22.895114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:22.895114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:22.895114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:22.895114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:22.895114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:22.895114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:22.895114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:22.895114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:22.895114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:22.895114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:45:22.895114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:45:22.895114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:45:22.895114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:45:22.897024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:22.897024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:22.897024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:45:22.897024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:22.897024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:45:22.897024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:22.897024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:22.897024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:22.897024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:22.897024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:22.897024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:22.897024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:22.897024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:22.897024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:22.897024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:22.897024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:22.897024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:22.897024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:22.898893 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:22.898893 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:22.898893 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:45:22.898893 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:45:22.898893 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:45:22.898893 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:45:22.898893 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:45:22.898893 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:45:22.898893 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:45:22.898893 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:22.898893 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 402/809] 10.222.0.16 () {56 vars in 1093 bytes} [Wed Aug 31 10:45:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:22.902992 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045217a-a261-4377-b3d0-aa607511d7fa None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:22.946764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:22.947780 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eafa4c2-669c-4893-9e9c-928ad4a79fba None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:22.995759 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c21dd-54d4-4e6e-a01d-8817b2c80394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:22.999790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c21dd-54d4-4e6e-a01d-8817b2c80394 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:23.000077 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c21dd-54d4-4e6e-a01d-8817b2c80394 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:23.000351 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c21dd-54d4-4e6e-a01d-8817b2c80394 None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:23.001045 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c21dd-54d4-4e6e-a01d-8817b2c80394 None admin] RBAC: Adding query filter params (name=service) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:23.001349 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c21dd-54d4-4e6e-a01d-8817b2c80394 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:23.003574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c21dd-54d4-4e6e-a01d-8817b2c80394 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:23.024526 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 408/810] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 31 10:45:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:23.031181 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c21dd-54d4-4e6e-a01d-8817b2c80394 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:23.087362 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:23.108653 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045217a-a261-4377-b3d0-aa607511d7fa None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:23.149683 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe980a-a104-4782-950e-e57ecc6d5928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:23.150304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe980a-a104-4782-950e-e57ecc6d5928 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:23.151855 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe980a-a104-4782-950e-e57ecc6d5928 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:23.151855 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe980a-a104-4782-950e-e57ecc6d5928 None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:23.153159 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe980a-a104-4782-950e-e57ecc6d5928 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:23.153518 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe980a-a104-4782-950e-e57ecc6d5928 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:23.154937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe980a-a104-4782-950e-e57ecc6d5928 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:23.163031 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c21dd-54d4-4e6e-a01d-8817b2c80394 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:23.176061 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 403/811] 10.222.0.16 () {56 vars in 1133 bytes} [Wed Aug 31 10:45:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:23.177314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7b55a-bc73-4bee-a4c5-353609f0f85d None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:23.177594 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7b55a-bc73-4bee-a4c5-353609f0f85d None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:23.177863 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7b55a-bc73-4bee-a4c5-353609f0f85d None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:23.179432 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 404/812] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:23.189250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104d1ba-cf54-4d37-a110-d5052ef859f0 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:23.190230 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104d1ba-cf54-4d37-a110-d5052ef859f0 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:23.190686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104d1ba-cf54-4d37-a110-d5052ef859f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:23.211940 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-3104d1ba-cf54-4d37-a110-d5052ef859f0 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:23.213141 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-3104d1ba-cf54-4d37-a110-d5052ef859f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:23.217582 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa88de44-81bc-4208-8bf9-4beb3ef7b7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:23.218440 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88de44-81bc-4208-8bf9-4beb3ef7b7d6 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:23.218868 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88de44-81bc-4208-8bf9-4beb3ef7b7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:23.219290 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88de44-81bc-4208-8bf9-4beb3ef7b7d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:23.234128 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa88de44-81bc-4208-8bf9-4beb3ef7b7d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:23.236546 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa88de44-81bc-4208-8bf9-4beb3ef7b7d6 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:23.257317 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-aa88de44-81bc-4208-8bf9-4beb3ef7b7d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:23.257317 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:23.257317 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:23.257317 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:23.257317 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:23.257317 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:23.257317 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:23.257317 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:23.257317 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:23.257317 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:23.257317 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:23.257317 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:23.257317 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:23.257317 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:23.257317 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:23.257317 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:23.269380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:23.269380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:23.269380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:23.269380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:23.269380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:23.269380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:23.269380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:23.269380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:23.269380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:23.269380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:23.269380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:23.269380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:23.269380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:23.269380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:23.269380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:23.269380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:23.269380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:23.270517 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:23.270517 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:23.270517 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:23.270517 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 409/813] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:23.270517 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa88de44-81bc-4208-8bf9-4beb3ef7b7d6 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:23.301070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 405/814] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:45:23.302649 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104d1ba-cf54-4d37-a110-d5052ef859f0 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:23.306495 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:23.422594 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104d1ba-cf54-4d37-a110-d5052ef859f0 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:23.467258 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7495a-a0c8-40bc-b033-1eb6f6d4eb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:23.468557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7495a-a0c8-40bc-b033-1eb6f6d4eb22 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:23.469037 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7495a-a0c8-40bc-b033-1eb6f6d4eb22 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:23.470181 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7495a-a0c8-40bc-b033-1eb6f6d4eb22 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:23.471070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f7495a-a0c8-40bc-b033-1eb6f6d4eb22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:23.471657 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f7495a-a0c8-40bc-b033-1eb6f6d4eb22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:23.473348 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f7495a-a0c8-40bc-b033-1eb6f6d4eb22 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:23.488884 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 406/815] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:23.495172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7495a-a0c8-40bc-b033-1eb6f6d4eb22 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:23.516524 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:23.521139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa88de44-81bc-4208-8bf9-4beb3ef7b7d6 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:23.594243 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:23.627672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a71def-5d78-4e8d-b3b4-af28b80278ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:23.628296 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a71def-5d78-4e8d-b3b4-af28b80278ec None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:23.628571 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a71def-5d78-4e8d-b3b4-af28b80278ec None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:23.628838 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a71def-5d78-4e8d-b3b4-af28b80278ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:23.634997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7495a-a0c8-40bc-b033-1eb6f6d4eb22 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:23.677529 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b5e746-567c-4017-9949-75d2b3d71cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:23.678199 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5e746-567c-4017-9949-75d2b3d71cd7 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:23.678499 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5e746-567c-4017-9949-75d2b3d71cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:23.678769 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5e746-567c-4017-9949-75d2b3d71cd7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:23.688289 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b5e746-567c-4017-9949-75d2b3d71cd7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:23.690370 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b5e746-567c-4017-9949-75d2b3d71cd7 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:23.706436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-65b5e746-567c-4017-9949-75d2b3d71cd7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 10:45:23.706436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:23.706436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:23.706436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:23.706436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:23.706436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:23.706436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:23.706436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:23.706436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:23.706436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:23.706436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:23.706436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:23.706436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:45:23.706436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:45:23.706436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:45:23.706436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:45:23.707442 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:23.707442 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:23.707442 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:45:23.707442 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:23.707442 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:45:23.707442 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:23.707442 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:23.707442 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:23.707442 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:23.707442 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:23.707442 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:23.707442 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:23.707442 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:23.707442 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:23.707442 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:23.707442 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:23.707442 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:23.707442 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:23.708981 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:23.708981 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:23.708981 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:45:23.708981 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:45:23.708981 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:45:23.708981 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:45:23.708981 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:45:23.708981 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:45:23.708981 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 10:45:23.708981 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:23.708981 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 407/816] 10.222.0.16 () {56 vars in 1117 bytes} [Wed Aug 31 10:45:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:23.709592 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b5e746-567c-4017-9949-75d2b3d71cd7 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:23.739819 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a71def-5d78-4e8d-b3b4-af28b80278ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:23.741416 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a71def-5d78-4e8d-b3b4-af28b80278ec None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:23.853739 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:23.858007 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b5e746-567c-4017-9949-75d2b3d71cd7 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:23.859422 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 410/817] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:45:23 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:45:23.868897 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a71def-5d78-4e8d-b3b4-af28b80278ec None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:23.892241 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:23.907015 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb5309-fddb-4d7b-bacc-c31d28413cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:23.907657 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb5309-fddb-4d7b-bacc-c31d28413cdb None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:23.908108 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb5309-fddb-4d7b-bacc-c31d28413cdb None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:23.908387 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb5309-fddb-4d7b-bacc-c31d28413cdb None admin] PATH_INFO: `/v3/services/network` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:23.909379 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb5309-fddb-4d7b-bacc-c31d28413cdb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:23.911179 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb5309-fddb-4d7b-bacc-c31d28413cdb None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:23.923984 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-3ceb5309-fddb-4d7b-bacc-c31d28413cdb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 10:45:23.923984 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:23.923984 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:45:23.923984 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:45:23.923984 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:45:23.923984 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:45:23.923984 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:45:23.923984 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:23.923984 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 10:45:23.923984 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:23.923984 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:45:23.923984 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:23.923984 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:23.923984 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:23.923984 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:23.923984 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:23.923984 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:23.925106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:23.925106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:23.925106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:23.925106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:23.925106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:23.925106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:23.925106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:45:23.925106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:45:23.925106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:45:23.925106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:45:23.925106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:23.925106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:23.925106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:23.925106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:23.925106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:23.925106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:23.926064 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:23.926064 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:23.926064 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:23.926064 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:23.926064 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:23.926064 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:23.926064 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:23.926064 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:23.926064 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:23.926064 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:23.926064 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:45:23.926064 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:23.926064 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 10:45:23.926064 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:23.926064 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 408/818] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:45:23 2022] GET /identity/v3/services/network => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:23.927963 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb5309-fddb-4d7b-bacc-c31d28413cdb None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:23.982009 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a71def-5d78-4e8d-b3b4-af28b80278ec None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:24.032587 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbae4ae-9f37-47e7-94f9-2b887e4d68bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:24.033826 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbae4ae-9f37-47e7-94f9-2b887e4d68bd None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:24.034828 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbae4ae-9f37-47e7-94f9-2b887e4d68bd None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:24.035339 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbae4ae-9f37-47e7-94f9-2b887e4d68bd None admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:24.036514 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbae4ae-9f37-47e7-94f9-2b887e4d68bd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:24.037274 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbae4ae-9f37-47e7-94f9-2b887e4d68bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:24.039433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbae4ae-9f37-47e7-94f9-2b887e4d68bd None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:24.059860 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 411/819] 10.222.0.16 () {56 vars in 1149 bytes} [Wed Aug 31 10:45:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:24.060864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbae4ae-9f37-47e7-94f9-2b887e4d68bd None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:24.062223 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb5309-fddb-4d7b-bacc-c31d28413cdb None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:24.112122 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace35b82-57e8-4747-ad96-9c3138c0a121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:24.113957 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace35b82-57e8-4747-ad96-9c3138c0a121 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:24.114443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace35b82-57e8-4747-ad96-9c3138c0a121 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:24.115001 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace35b82-57e8-4747-ad96-9c3138c0a121 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:24.130665 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace35b82-57e8-4747-ad96-9c3138c0a121 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:24.133285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace35b82-57e8-4747-ad96-9c3138c0a121 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:24.142918 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:24.152400 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-ace35b82-57e8-4747-ad96-9c3138c0a121 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:24.152400 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:24.152400 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:24.152400 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:24.152400 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:24.152400 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:24.152400 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:24.152400 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:24.152400 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:24.152400 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:24.152400 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:24.152400 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:24.152400 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:24.152400 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:24.152400 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:24.152400 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:24.153453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:24.153453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:24.153453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:24.153453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:24.153453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:24.153453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:24.153453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:24.153453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:24.153453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:24.153453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:24.153453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:24.153453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:24.153453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:24.153453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:24.153453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:24.153453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:24.153453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:24.154325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:24.154325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:24.154325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:24.155972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 409/820] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:24.157023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace35b82-57e8-4747-ad96-9c3138c0a121 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:24.186618 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbae4ae-9f37-47e7-94f9-2b887e4d68bd None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:24.206817 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:24.232000 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbfcc13-0b6c-4f6a-8544-0c5cd0670d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:24.234931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfcc13-0b6c-4f6a-8544-0c5cd0670d8c None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:24.234931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfcc13-0b6c-4f6a-8544-0c5cd0670d8c None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:24.234931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfcc13-0b6c-4f6a-8544-0c5cd0670d8c None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:24.234931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbfcc13-0b6c-4f6a-8544-0c5cd0670d8c None admin] RBAC: Adding query filter params (name=network) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:24.234931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbfcc13-0b6c-4f6a-8544-0c5cd0670d8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:24.235552 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbfcc13-0b6c-4f6a-8544-0c5cd0670d8c None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:24.240851 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 412/821] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Aug 31 10:45:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:24.241822 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbfcc13-0b6c-4f6a-8544-0c5cd0670d8c None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:24.290702 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace35b82-57e8-4747-ad96-9c3138c0a121 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:24.333597 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f14180-4b47-441e-9aef-b3d70160b333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:24.334722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f14180-4b47-441e-9aef-b3d70160b333 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:24.335209 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f14180-4b47-441e-9aef-b3d70160b333 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:24.335700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f14180-4b47-441e-9aef-b3d70160b333 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:24.349375 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f14180-4b47-441e-9aef-b3d70160b333 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:24.350987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f14180-4b47-441e-9aef-b3d70160b333 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:24.366757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-13f14180-4b47-441e-9aef-b3d70160b333 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:24.366757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:24.366757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:24.366757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:24.366757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:24.366757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:24.366757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:24.366757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:24.366757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:24.366757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:24.366757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:24.366757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:24.366757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:24.366757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:24.366757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:24.366757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:24.367796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:24.367796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:24.367796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:24.367796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:24.367796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:24.367796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:24.367796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:24.367796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:24.367796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:24.367796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:24.367796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:24.367796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:24.367796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:24.367796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:24.367796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:24.367796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:24.367796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:24.368717 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:24.368717 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:24.368717 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:24.368896 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbfcc13-0b6c-4f6a-8544-0c5cd0670d8c None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:24.368971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 410/822] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:24.368971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f14180-4b47-441e-9aef-b3d70160b333 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:24.392132 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:24.408278 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc040f9-fc45-4a3d-b040-316977e618f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:24.409240 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc040f9-fc45-4a3d-b040-316977e618f3 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:24.409521 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc040f9-fc45-4a3d-b040-316977e618f3 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:24.409830 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc040f9-fc45-4a3d-b040-316977e618f3 None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:24.410466 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc040f9-fc45-4a3d-b040-316977e618f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:24.410776 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc040f9-fc45-4a3d-b040-316977e618f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:24.412196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc040f9-fc45-4a3d-b040-316977e618f3 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:24.429363 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 413/823] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:24.433083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6c992-2707-435a-b4e6-5942432e404b None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:24.433083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6c992-2707-435a-b4e6-5942432e404b None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:24.433083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6c992-2707-435a-b4e6-5942432e404b None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:24.433083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 414/824] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:24.433403 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6c992-2707-435a-b4e6-5942432e404b None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:24.485538 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:24.494803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f14180-4b47-441e-9aef-b3d70160b333 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:24.531023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ea3da-3dff-4c11-ba4f-f2ac26e7d9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:24.531629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ea3da-3dff-4c11-ba4f-f2ac26e7d9ba None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:24.531901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ea3da-3dff-4c11-ba4f-f2ac26e7d9ba None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:24.532168 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ea3da-3dff-4c11-ba4f-f2ac26e7d9ba None admin] PATH_INFO: `/v3/services` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:24.532793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ea3da-3dff-4c11-ba4f-f2ac26e7d9ba None admin] RBAC: Adding query filter params (type=network) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:24.533095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ea3da-3dff-4c11-ba4f-f2ac26e7d9ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:24.534600 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ea3da-3dff-4c11-ba4f-f2ac26e7d9ba None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:24.539936 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 411/825] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Aug 31 10:45:24 2022] GET /identity/v3/services?type=network => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:24.541485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ea3da-3dff-4c11-ba4f-f2ac26e7d9ba None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:24.554667 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6c992-2707-435a-b4e6-5942432e404b None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:24.594349 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b8df2-b2c9-40d5-9501-77734664440f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:24.595240 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b8df2-b2c9-40d5-9501-77734664440f None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:24.595531 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b8df2-b2c9-40d5-9501-77734664440f None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:24.595799 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b8df2-b2c9-40d5-9501-77734664440f None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:24.596433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66b8df2-b2c9-40d5-9501-77734664440f None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:24.596729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66b8df2-b2c9-40d5-9501-77734664440f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:24.598103 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66b8df2-b2c9-40d5-9501-77734664440f None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:24.614856 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 415/826] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:24.614856 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c791ca-b5e8-45a2-b6e4-16516032f317 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:24.614856 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c791ca-b5e8-45a2-b6e4-16516032f317 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:24.614856 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c791ca-b5e8-45a2-b6e4-16516032f317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:24.632461 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-63c791ca-b5e8-45a2-b6e4-16516032f317 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:24.633450 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-63c791ca-b5e8-45a2-b6e4-16516032f317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:24.660465 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ea3da-3dff-4c11-ba4f-f2ac26e7d9ba None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:24.675941 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:24.704185 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56444e9f-22c3-446c-8c8a-1c9befe6411b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:24.704824 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56444e9f-22c3-446c-8c8a-1c9befe6411b None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:24.705098 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56444e9f-22c3-446c-8c8a-1c9befe6411b None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:24.705367 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56444e9f-22c3-446c-8c8a-1c9befe6411b None admin] PATH_INFO: `/v3/projects/service` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:24.712413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56444e9f-22c3-446c-8c8a-1c9befe6411b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:24.722574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 416/827] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:45:24.722729 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56444e9f-22c3-446c-8c8a-1c9befe6411b None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:24.722819 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c791ca-b5e8-45a2-b6e4-16516032f317 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:24.735502 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-56444e9f-22c3-446c-8c8a-1c9befe6411b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:24.735502 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:24.735502 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:24.735502 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:24.735502 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:24.735502 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:24.735502 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:24.735502 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:24.735502 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:24.735502 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:24.735502 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:24.735502 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:24.735502 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:45:24.735502 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:45:24.735502 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:45:24.736488 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:45:24.736488 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:24.736488 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:24.736488 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:24.736488 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:24.736488 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:24.736488 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:24.736488 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:24.736488 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:24.736488 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:24.736488 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:24.736488 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:24.736488 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:24.736488 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:24.736488 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:24.736488 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:24.736488 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:24.737366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:45:24.737366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:45:24.737366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:45:24.737366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:45:24.737366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:45:24.737366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:45:24.737366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:24.737366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:24.737366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 412/828] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:45:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:24.737839 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56444e9f-22c3-446c-8c8a-1c9befe6411b None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:24.770830 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:24.869486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c791ca-b5e8-45a2-b6e4-16516032f317 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:24.872050 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56444e9f-22c3-446c-8c8a-1c9befe6411b None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:24.909132 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45db577-67c0-4449-b8c9-cf489c6a0b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:24.909860 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45db577-67c0-4449-b8c9-cf489c6a0b93 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:24.910247 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45db577-67c0-4449-b8c9-cf489c6a0b93 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:24.910598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45db577-67c0-4449-b8c9-cf489c6a0b93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:24.916179 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d395bf5b-524a-4ce8-9a04-1b20cb253d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:24.916795 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d395bf5b-524a-4ce8-9a04-1b20cb253d37 None admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:24.917357 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d395bf5b-524a-4ce8-9a04-1b20cb253d37 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:24.918173 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d395bf5b-524a-4ce8-9a04-1b20cb253d37 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:24.918779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45db577-67c0-4449-b8c9-cf489c6a0b93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:24.918934 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d395bf5b-524a-4ce8-9a04-1b20cb253d37 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:24.920358 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d395bf5b-524a-4ce8-9a04-1b20cb253d37 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:24.920521 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45db577-67c0-4449-b8c9-cf489c6a0b93 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:24.932783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-b45db577-67c0-4449-b8c9-cf489c6a0b93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:24.932783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:24.932783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:24.932783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:24.932783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:24.932783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:24.932783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:24.932783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:24.932783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:24.932783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:24.932783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:24.932783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:24.932783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:45:24.932783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:45:24.932783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:45:24.934761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:45:24.934761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:24.934761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:24.934761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:24.934761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:24.934761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:24.934761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:24.934761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:24.934761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:24.934761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:24.934761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:24.934761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:24.934761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:24.934761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:24.934761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:24.934761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:24.934761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:24.935646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:45:24.935646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:45:24.935646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:45:24.935646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:45:24.935646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:45:24.935646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:45:24.935646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:24.935646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:24.938533 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 413/829] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:45:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:24.939599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45db577-67c0-4449-b8c9-cf489c6a0b93 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:25.021599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:25.029569 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 417/830] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 10:45:24 2022] POST /identity/v3/endpoints => generated 337 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:45:25.044007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d395bf5b-524a-4ce8-9a04-1b20cb253d37 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:25.048591 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45db577-67c0-4449-b8c9-cf489c6a0b93 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:25.091131 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:25.109491 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a925d39c-3de1-4e97-a6f4-dd57f81c7520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:25.110144 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925d39c-3de1-4e97-a6f4-dd57f81c7520 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:25.110447 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925d39c-3de1-4e97-a6f4-dd57f81c7520 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:25.110779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925d39c-3de1-4e97-a6f4-dd57f81c7520 None admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:25.111510 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a925d39c-3de1-4e97-a6f4-dd57f81c7520 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:25.111888 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a925d39c-3de1-4e97-a6f4-dd57f81c7520 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:25.113770 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a925d39c-3de1-4e97-a6f4-dd57f81c7520 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:25.128923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 414/831] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:45:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:25.129817 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a925d39c-3de1-4e97-a6f4-dd57f81c7520 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:25.191180 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d395bf5b-524a-4ce8-9a04-1b20cb253d37 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:25.237081 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b9a4c-386c-4e18-99fb-d6cc43a2b8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:25.237705 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b9a4c-386c-4e18-99fb-d6cc43a2b8d0 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:25.237983 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b9a4c-386c-4e18-99fb-d6cc43a2b8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:25.238263 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b9a4c-386c-4e18-99fb-d6cc43a2b8d0 None admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:25.239485 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b9a4c-386c-4e18-99fb-d6cc43a2b8d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:25.239784 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b9a4c-386c-4e18-99fb-d6cc43a2b8d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:25.241254 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b9a4c-386c-4e18-99fb-d6cc43a2b8d0 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:25.254855 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 418/832] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:45:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:25.255961 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e553ffd-e519-4aa0-b75e-7261cb5ec08f None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:25.256251 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e553ffd-e519-4aa0-b75e-7261cb5ec08f None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:25.256545 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e553ffd-e519-4aa0-b75e-7261cb5ec08f None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:25.257807 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 419/833] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:25.260526 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e553ffd-e519-4aa0-b75e-7261cb5ec08f None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:25.274138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a925d39c-3de1-4e97-a6f4-dd57f81c7520 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:25.305387 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:25.311581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055aaef-05f3-4527-bf04-8ef7c1ee7546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:25.312220 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055aaef-05f3-4527-bf04-8ef7c1ee7546 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:25.312493 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055aaef-05f3-4527-bf04-8ef7c1ee7546 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:25.312763 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055aaef-05f3-4527-bf04-8ef7c1ee7546 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:25.323124 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d055aaef-05f3-4527-bf04-8ef7c1ee7546 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:45:25.328704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-d055aaef-05f3-4527-bf04-8ef7c1ee7546 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:45:25.328704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:25.328704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:25.328704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:25.328704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:25.328704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:25.328704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:25.328704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:25.328704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:25.328704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:25.328704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:25.328704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:25.328704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:45:25.328704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:45:25.328704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:45:25.328704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:45:25.329768 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:45:25.329768 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:45:25.329768 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:45:25.329768 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:45:25.329768 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:25.329768 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:25.329768 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:25.329768 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:25.329768 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:25.329768 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:25.329768 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:25.329768 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:25.329768 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:25.329768 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:25.329768 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:25.329768 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:25.330614 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:25.330614 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:25.330614 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:25.330614 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:25.330614 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:45:25.330614 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:45:25.330614 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:45:25.330614 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:45:25.330614 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:45:25.330614 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:45:25.330614 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:45:25.330614 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:25.330614 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 415/834] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:45:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:25.330614 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055aaef-05f3-4527-bf04-8ef7c1ee7546 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:25.354027 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:25.371701 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e553ffd-e519-4aa0-b75e-7261cb5ec08f None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:25.407722 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4cf8af-0b4d-4ba5-af72-1336c30ec6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:25.408327 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cf8af-0b4d-4ba5-af72-1336c30ec6b2 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:25.408601 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cf8af-0b4d-4ba5-af72-1336c30ec6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:25.408884 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cf8af-0b4d-4ba5-af72-1336c30ec6b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:25.409597 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4cf8af-0b4d-4ba5-af72-1336c30ec6b2 None admin] RBAC: Adding query filter params (role.id=13ca33d694c645d6920930d39dc52a0d, scope.project.id=8db9b3f8a9f041938ac7b39003a35015, user.id=f11971a594014d86968c3356d11d42f7) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:25.409899 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4cf8af-0b4d-4ba5-af72-1336c30ec6b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:25.411343 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4cf8af-0b4d-4ba5-af72-1336c30ec6b2 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:25.417252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 420/835] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:45:25 2022] GET /identity/v3/role_assignments?user.id=f11971a594014d86968c3356d11d42f7&scope.project.id=8db9b3f8a9f041938ac7b39003a35015&role.id=13ca33d694c645d6920930d39dc52a0d => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:25.418880 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6919c-80c3-4903-bcd3-68ffad1b3968 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:25.419169 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6919c-80c3-4903-bcd3-68ffad1b3968 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:25.419439 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6919c-80c3-4903-bcd3-68ffad1b3968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:25.439198 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-a2e6919c-80c3-4903-bcd3-68ffad1b3968 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:25.440437 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a2e6919c-80c3-4903-bcd3-68ffad1b3968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:25.452467 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055aaef-05f3-4527-bf04-8ef7c1ee7546 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:25.498540 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa878b5-d191-432e-b9ba-4d1e8f7954bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:25.499262 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa878b5-d191-432e-b9ba-4d1e8f7954bd None admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:25.499538 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa878b5-d191-432e-b9ba-4d1e8f7954bd None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:25.499865 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa878b5-d191-432e-b9ba-4d1e8f7954bd None admin] PATH_INFO: `/v3/projects/8db9b3f8a9f041938ac7b39003a35015/users/7796ebff70054de090e02c58bf7000f4/roles/bec755d6565a41798b09f5f57c1d51dd` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:25.517512 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa878b5-d191-432e-b9ba-4d1e8f7954bd None admin] RBAC: Authorizing `identity:create_grant(project_id=8db9b3f8a9f041938ac7b39003a35015, user_id=7796ebff70054de090e02c58bf7000f4, role_id=bec755d6565a41798b09f5f57c1d51dd)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:25.519013 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa878b5-d191-432e-b9ba-4d1e8f7954bd None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:25.521809 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 421/836] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:45:25.522630 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6919c-80c3-4903-bcd3-68ffad1b3968 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:25.603680 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 416/837] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 31 10:45:25 2022] PUT /identity/v3/projects/8db9b3f8a9f041938ac7b39003a35015/users/7796ebff70054de090e02c58bf7000f4/roles/bec755d6565a41798b09f5f57c1d51dd => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:45:25.626141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:25.654797 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6919c-80c3-4903-bcd3-68ffad1b3968 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:25.700315 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64830b97-356b-4804-a902-a3037d30bd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:25.700960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64830b97-356b-4804-a902-a3037d30bd7d None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:25.701241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64830b97-356b-4804-a902-a3037d30bd7d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:25.701513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64830b97-356b-4804-a902-a3037d30bd7d None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:25.702199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64830b97-356b-4804-a902-a3037d30bd7d None admin] RBAC: Adding query filter params (name=admin) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:25.703359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64830b97-356b-4804-a902-a3037d30bd7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:25.703956 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64830b97-356b-4804-a902-a3037d30bd7d None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:25.716639 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 422/838] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:45:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:25.721980 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa878b5-d191-432e-b9ba-4d1e8f7954bd None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:25.773190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:25.822550 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64830b97-356b-4804-a902-a3037d30bd7d None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:25.849471 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa878b5-d191-432e-b9ba-4d1e8f7954bd None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:25.899320 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b2be8-ba78-4f80-bc99-bab9cac94191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:25.899875 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b2be8-ba78-4f80-bc99-bab9cac94191 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:25.900149 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b2be8-ba78-4f80-bc99-bab9cac94191 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:25.900475 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b2be8-ba78-4f80-bc99-bab9cac94191 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:25.914245 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b2be8-ba78-4f80-bc99-bab9cac94191 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:25.915739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b2be8-ba78-4f80-bc99-bab9cac94191 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:25.932843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-628b2be8-ba78-4f80-bc99-bab9cac94191 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:25.932843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:25.932843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:25.932843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:25.932843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:25.932843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:25.932843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:25.932843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:25.932843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:25.932843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:25.932843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:25.932843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:25.932843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:25.932843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:25.932843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:25.932843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:25.934536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:25.934536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:25.934536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:25.934536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:25.934536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:25.934536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:25.934536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:25.934536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:25.934536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:25.934536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:25.934536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:25.934536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:25.934536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:25.934536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:25.934536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:25.934536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:25.934536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:25.935407 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:25.935407 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:25.935407 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:25.935572 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 417/839] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:25.941344 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b2be8-ba78-4f80-bc99-bab9cac94191 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:25.951892 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64830b97-356b-4804-a902-a3037d30bd7d None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:25.991314 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38bc65-cdf4-4a89-ad19-319efd508afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:25.992066 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38bc65-cdf4-4a89-ad19-319efd508afc None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:25.992345 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38bc65-cdf4-4a89-ad19-319efd508afc None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:25.992612 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38bc65-cdf4-4a89-ad19-319efd508afc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:25.993194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f38bc65-cdf4-4a89-ad19-319efd508afc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:25.995053 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f38bc65-cdf4-4a89-ad19-319efd508afc None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:26.005814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-0f38bc65-cdf4-4a89-ad19-319efd508afc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 10:45:26.005814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:26.005814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:45:26.005814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:45:26.005814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:45:26.005814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:45:26.005814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:45:26.005814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:26.005814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 10:45:26.005814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:26.005814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:45:26.005814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:26.005814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:26.005814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:26.005814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:26.005814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:26.005814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:26.007080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:26.007080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:26.007080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:26.007080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:26.007080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:26.007080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:26.007080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:45:26.007080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:45:26.007080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:45:26.007080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:45:26.007080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:26.007080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:26.007080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:26.007080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:26.007080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:26.007080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:26.007916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:26.007916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:26.007916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:26.007916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:26.007916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:26.007916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:26.007916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:26.007916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:26.007916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:26.007916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:26.007916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:45:26.007916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:26.007916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 10:45:26.007916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:26.011971 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 423/840] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 31 10:45:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:26.014694 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38bc65-cdf4-4a89-ad19-319efd508afc None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:26.025126 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:26.050177 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:26.057705 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b2be8-ba78-4f80-bc99-bab9cac94191 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:26.105719 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd8b8e-a003-413b-b027-7ad3cc5ca6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:26.105719 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd8b8e-a003-413b-b027-7ad3cc5ca6d5 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:26.105719 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd8b8e-a003-413b-b027-7ad3cc5ca6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:26.105719 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd8b8e-a003-413b-b027-7ad3cc5ca6d5 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:26.106114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd8b8e-a003-413b-b027-7ad3cc5ca6d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:26.106114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd8b8e-a003-413b-b027-7ad3cc5ca6d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:26.106843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd8b8e-a003-413b-b027-7ad3cc5ca6d5 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:26.120565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 418/841] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:26.126566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd8b8e-a003-413b-b027-7ad3cc5ca6d5 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:26.152885 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38bc65-cdf4-4a89-ad19-319efd508afc None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:26.191451 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd789d72-b3af-4551-af01-b510f0bc78e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:26.192306 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd789d72-b3af-4551-af01-b510f0bc78e1 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:26.192652 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd789d72-b3af-4551-af01-b510f0bc78e1 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:26.192989 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd789d72-b3af-4551-af01-b510f0bc78e1 None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:26.193965 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd789d72-b3af-4551-af01-b510f0bc78e1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:26.194331 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd789d72-b3af-4551-af01-b510f0bc78e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:26.195797 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd789d72-b3af-4551-af01-b510f0bc78e1 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:26.201854 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 424/842] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 31 10:45:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:26.207226 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd789d72-b3af-4551-af01-b510f0bc78e1 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:26.238289 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd8b8e-a003-413b-b027-7ad3cc5ca6d5 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:26.283767 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c42b8-48a1-464f-b3ad-a6d559ec380e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:26.284439 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c42b8-48a1-464f-b3ad-a6d559ec380e None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:26.284978 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c42b8-48a1-464f-b3ad-a6d559ec380e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:26.285258 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c42b8-48a1-464f-b3ad-a6d559ec380e None admin] PATH_INFO: `/v3/users/swift` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:26.297567 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c42b8-48a1-464f-b3ad-a6d559ec380e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:26.299669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c42b8-48a1-464f-b3ad-a6d559ec380e None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:26.304463 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:26.318189 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-d61c42b8-48a1-464f-b3ad-a6d559ec380e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 10:45:26.318189 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:26.318189 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:26.318189 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:26.318189 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:26.318189 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:26.318189 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:26.318189 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:26.318189 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:26.318189 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:26.318189 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:26.318189 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:26.318189 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:45:26.318189 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:45:26.318189 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:45:26.318189 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:45:26.319186 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:26.319186 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:26.319186 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:45:26.319186 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:26.319186 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:45:26.319186 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:26.319186 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:26.319186 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:26.319186 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:26.319186 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:26.319186 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:26.319186 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:26.319186 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:26.319186 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:26.319186 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:26.319186 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:26.319186 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:26.319186 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:26.320123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:26.320123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:26.320123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:45:26.320123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:45:26.320123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:45:26.320123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:45:26.320123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:45:26.320123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:45:26.320123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 10:45:26.320123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:26.322458 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 419/843] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:45:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:26.328246 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c42b8-48a1-464f-b3ad-a6d559ec380e None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:26.348194 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:26.361772 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd789d72-b3af-4551-af01-b510f0bc78e1 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:26.413324 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656989b-bd22-4b8b-b4bd-e3da3a1cf8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:26.414062 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656989b-bd22-4b8b-b4bd-e3da3a1cf8a4 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:26.414466 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656989b-bd22-4b8b-b4bd-e3da3a1cf8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:26.414855 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656989b-bd22-4b8b-b4bd-e3da3a1cf8a4 None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:26.415526 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656989b-bd22-4b8b-b4bd-e3da3a1cf8a4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:26.415893 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656989b-bd22-4b8b-b4bd-e3da3a1cf8a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:26.417332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656989b-bd22-4b8b-b4bd-e3da3a1cf8a4 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:26.429751 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 425/844] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 31 10:45:26 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:26.455154 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c42b8-48a1-464f-b3ad-a6d559ec380e None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:26.497206 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8701f-70fc-45ed-9606-e338fd74c128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:26.498177 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8701f-70fc-45ed-9606-e338fd74c128 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:26.498668 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8701f-70fc-45ed-9606-e338fd74c128 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:26.499194 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8701f-70fc-45ed-9606-e338fd74c128 None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:26.500122 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e8701f-70fc-45ed-9606-e338fd74c128 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:26.500610 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e8701f-70fc-45ed-9606-e338fd74c128 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:26.502265 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e8701f-70fc-45ed-9606-e338fd74c128 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:26.524346 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 420/845] 10.222.0.16 () {56 vars in 1135 bytes} [Wed Aug 31 10:45:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:26.533005 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656989b-bd22-4b8b-b4bd-e3da3a1cf8a4 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:26.645903 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656989b-bd22-4b8b-b4bd-e3da3a1cf8a4 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:26.682846 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b9073-3ff5-4848-a92f-76d2ad0a6fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:26.684154 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b9073-3ff5-4848-a92f-76d2ad0a6fa9 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:26.684425 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b9073-3ff5-4848-a92f-76d2ad0a6fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:26.684689 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b9073-3ff5-4848-a92f-76d2ad0a6fa9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:26.687664 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:26.766748 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b9073-3ff5-4848-a92f-76d2ad0a6fa9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:26.768280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b9073-3ff5-4848-a92f-76d2ad0a6fa9 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:26.877396 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 426/846] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:45:26 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:45:26.882716 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8701f-70fc-45ed-9606-e338fd74c128 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:26.956748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:26.992262 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8701f-70fc-45ed-9606-e338fd74c128 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:27.036500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2240b8-a932-4cda-bede-afd54d7444d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:27.037133 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2240b8-a932-4cda-bede-afd54d7444d4 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:27.037444 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2240b8-a932-4cda-bede-afd54d7444d4 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:27.037715 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2240b8-a932-4cda-bede-afd54d7444d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:27.053218 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2240b8-a932-4cda-bede-afd54d7444d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:27.054701 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2240b8-a932-4cda-bede-afd54d7444d4 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:27.069952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-eb2240b8-a932-4cda-bede-afd54d7444d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:27.069952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:27.069952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:27.069952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:27.069952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:27.069952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:27.069952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:27.069952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:27.069952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:27.069952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:27.069952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:27.069952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:27.069952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:27.069952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:27.069952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:27.069952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:27.070992 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:27.070992 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:27.070992 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:27.070992 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:27.070992 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:27.070992 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:27.070992 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:27.070992 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:27.070992 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:27.070992 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:27.070992 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:27.070992 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:27.070992 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:27.070992 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:27.070992 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:27.070992 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:27.070992 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:27.071877 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:27.071877 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:27.071877 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:27.071877 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 421/847] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:27.082450 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b9073-3ff5-4848-a92f-76d2ad0a6fa9 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:27.120223 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:27.197028 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b9073-3ff5-4848-a92f-76d2ad0a6fa9 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:27.235857 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0933b-577c-4a97-93e9-58d2dff67283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:27.236457 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0933b-577c-4a97-93e9-58d2dff67283 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:27.236742 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0933b-577c-4a97-93e9-58d2dff67283 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:27.237014 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0933b-577c-4a97-93e9-58d2dff67283 None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:27.237663 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f0933b-577c-4a97-93e9-58d2dff67283 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:27.237966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f0933b-577c-4a97-93e9-58d2dff67283 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:27.239356 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f0933b-577c-4a97-93e9-58d2dff67283 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:27.253271 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 427/848] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:27.259557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2240b8-a932-4cda-bede-afd54d7444d4 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:27.372904 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2240b8-a932-4cda-bede-afd54d7444d4 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:27.401885 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:27.411420 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de259f-8960-4945-95bf-cc5727a0367c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:27.412277 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de259f-8960-4945-95bf-cc5727a0367c None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:27.412735 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de259f-8960-4945-95bf-cc5727a0367c None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:27.413154 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de259f-8960-4945-95bf-cc5727a0367c None admin] PATH_INFO: `/v3/projects/service` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:27.420745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de259f-8960-4945-95bf-cc5727a0367c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:27.422479 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de259f-8960-4945-95bf-cc5727a0367c None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:27.434940 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-88de259f-8960-4945-95bf-cc5727a0367c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:27.434940 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:27.434940 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:27.434940 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:27.434940 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:27.434940 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:27.434940 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:27.434940 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:27.434940 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:27.434940 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:27.434940 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:27.434940 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:27.434940 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:45:27.434940 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:45:27.434940 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:45:27.436156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:45:27.436156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:27.436156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:27.436156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:27.436156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:27.436156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:27.436156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:27.436156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:27.436156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:27.436156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:27.436156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:27.436156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:27.436156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:27.436156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:27.436156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:27.436156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:27.436156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:27.437250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:45:27.437250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:45:27.437250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:45:27.437250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:45:27.437250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:45:27.437250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:45:27.437250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:27.437250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:27.439326 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 422/849] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:45:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:27.446479 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0933b-577c-4a97-93e9-58d2dff67283 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:27.554217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:27.562431 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0933b-577c-4a97-93e9-58d2dff67283 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:27.593279 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11795a2-0ab6-425b-a0d8-f7173ed2d002 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:27.593652 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11795a2-0ab6-425b-a0d8-f7173ed2d002 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:27.593835 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11795a2-0ab6-425b-a0d8-f7173ed2d002 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:27.595360 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 423/850] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:45:27.600416 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1017a-5878-4120-8b8a-8a4b7ecfdc64 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:27.600702 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1017a-5878-4120-8b8a-8a4b7ecfdc64 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:27.600982 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1017a-5878-4120-8b8a-8a4b7ecfdc64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:27.607559 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d288c0-6729-4c22-bfe3-bb7ce29a6e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:27.608493 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d288c0-6729-4c22-bfe3-bb7ce29a6e8b None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:27.609069 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d288c0-6729-4c22-bfe3-bb7ce29a6e8b None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:27.609553 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d288c0-6729-4c22-bfe3-bb7ce29a6e8b None admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:27.610504 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d288c0-6729-4c22-bfe3-bb7ce29a6e8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:27.611010 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d288c0-6729-4c22-bfe3-bb7ce29a6e8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:27.612658 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d288c0-6729-4c22-bfe3-bb7ce29a6e8b None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:27.621359 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-f2f1017a-5878-4120-8b8a-8a4b7ecfdc64 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:27.622674 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-f2f1017a-5878-4120-8b8a-8a4b7ecfdc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:27.628251 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 428/851] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:45:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:27.635554 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d288c0-6729-4c22-bfe3-bb7ce29a6e8b None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:27.699828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 424/852] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:27 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:45:27.701202 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24145ab6-729a-4cf5-b808-4234264fb812 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:27.701493 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24145ab6-729a-4cf5-b808-4234264fb812 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:27.701765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24145ab6-729a-4cf5-b808-4234264fb812 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:27.707387 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 425/853] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:27.708704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12c1cb-e2ee-4057-b5b3-94054ff5c16a None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:27.708987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12c1cb-e2ee-4057-b5b3-94054ff5c16a None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:27.709270 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12c1cb-e2ee-4057-b5b3-94054ff5c16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:27.729591 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-aa12c1cb-e2ee-4057-b5b3-94054ff5c16a None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:27.732185 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-aa12c1cb-e2ee-4057-b5b3-94054ff5c16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:27.761835 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d288c0-6729-4c22-bfe3-bb7ce29a6e8b None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:27.802565 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab524f-f597-44f6-91c1-1036e67994c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:27.803222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab524f-f597-44f6-91c1-1036e67994c8 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:27.803432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab524f-f597-44f6-91c1-1036e67994c8 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:27.803725 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab524f-f597-44f6-91c1-1036e67994c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:27.804505 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab524f-f597-44f6-91c1-1036e67994c8 None admin] RBAC: Adding query filter params (role.id=13ca33d694c645d6920930d39dc52a0d, scope.project.id=8db9b3f8a9f041938ac7b39003a35015, user.id=e8abc7c439a04622896b34ed0a7b8492) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:27.804757 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab524f-f597-44f6-91c1-1036e67994c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:27.806195 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab524f-f597-44f6-91c1-1036e67994c8 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:27.818578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 429/854] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:45:27 2022] GET /identity/v3/role_assignments?user.id=e8abc7c439a04622896b34ed0a7b8492&scope.project.id=8db9b3f8a9f041938ac7b39003a35015&role.id=13ca33d694c645d6920930d39dc52a0d => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:27.821241 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 426/855] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:27 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:45:27.976563 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab524f-f597-44f6-91c1-1036e67994c8 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:27.998628 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:28.100980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab524f-f597-44f6-91c1-1036e67994c8 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:28.117103 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12c1cb-e2ee-4057-b5b3-94054ff5c16a None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:28.148402 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7fee14-6b32-4745-b6b1-70d225373faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:28.149497 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fee14-6b32-4745-b6b1-70d225373faa None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:28.149992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fee14-6b32-4745-b6b1-70d225373faa None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:28.150561 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fee14-6b32-4745-b6b1-70d225373faa None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:28.151367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7fee14-6b32-4745-b6b1-70d225373faa None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:28.153712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7fee14-6b32-4745-b6b1-70d225373faa None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:28.163774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-fd7fee14-6b32-4745-b6b1-70d225373faa None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 10:45:28.163774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:28.163774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:45:28.163774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:45:28.163774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:45:28.163774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:45:28.163774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:45:28.163774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:28.163774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 10:45:28.163774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:28.163774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:45:28.163774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:28.163774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:28.163774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:28.163774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:28.163774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:28.164753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:28.164753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:28.164753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:28.164753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:28.164753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:28.164753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:28.164753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:28.164753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:45:28.164753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:45:28.164753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:45:28.164753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:45:28.164753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:28.164753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:28.164753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:28.164753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:28.164753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:28.164753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:28.165634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:28.165634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:28.165634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:28.165634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:28.165634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:28.165634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:28.165634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:28.165634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:28.165634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:28.165634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:28.165634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:45:28.165634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:28.165634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 10:45:28.165634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:28.168039 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:28.170546 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 430/856] 10.222.0.16 () {56 vars in 1132 bytes} [Wed Aug 31 10:45:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:28.176725 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7fee14-6b32-4745-b6b1-70d225373faa None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:28.236299 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12c1cb-e2ee-4057-b5b3-94054ff5c16a None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:28.265940 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:28.275483 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b2f744-7dc8-4e2e-963a-d86ed8398ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:28.276111 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2f744-7dc8-4e2e-963a-d86ed8398ad2 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:28.276385 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2f744-7dc8-4e2e-963a-d86ed8398ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:28.278337 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2f744-7dc8-4e2e-963a-d86ed8398ad2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:28.288406 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7fee14-6b32-4745-b6b1-70d225373faa None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:28.289286 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b2f744-7dc8-4e2e-963a-d86ed8398ad2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:45:28.294673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-a4b2f744-7dc8-4e2e-963a-d86ed8398ad2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:45:28.294673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:28.294673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:28.294673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:28.294673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:28.294673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:28.294673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:28.294673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:28.294673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:28.294673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:28.294673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:28.294673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:28.294673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:45:28.294673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:45:28.294673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:45:28.294673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:45:28.295882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:45:28.295882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:45:28.295882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:45:28.295882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:45:28.295882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:28.295882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:28.295882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:28.295882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:28.295882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:28.295882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:28.295882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:28.295882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:28.295882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:28.295882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:28.295882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:28.295882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:28.296896 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:28.296896 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:28.296896 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:28.296896 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:28.296896 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:45:28.296896 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:45:28.296896 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:45:28.296896 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:45:28.296896 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:45:28.296896 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:45:28.296896 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:45:28.296896 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:28.298267 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 427/857] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 31 10:45:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:28.304268 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b2f744-7dc8-4e2e-963a-d86ed8398ad2 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:28.332554 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935acfb-9fa3-4c1a-bd00-79f4c0c4dac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:28.333415 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935acfb-9fa3-4c1a-bd00-79f4c0c4dac6 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:28.333863 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935acfb-9fa3-4c1a-bd00-79f4c0c4dac6 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:28.334298 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935acfb-9fa3-4c1a-bd00-79f4c0c4dac6 None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:28.335113 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5935acfb-9fa3-4c1a-bd00-79f4c0c4dac6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:28.335583 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5935acfb-9fa3-4c1a-bd00-79f4c0c4dac6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:28.337169 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5935acfb-9fa3-4c1a-bd00-79f4c0c4dac6 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:28.348152 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 431/858] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Aug 31 10:45:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:28.349411 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935acfb-9fa3-4c1a-bd00-79f4c0c4dac6 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:28.445871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b2f744-7dc8-4e2e-963a-d86ed8398ad2 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:28.448957 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:28.473430 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935acfb-9fa3-4c1a-bd00-79f4c0c4dac6 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:28.488769 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09c688-b41f-4e26-b52b-8a3917f7cb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:28.489392 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09c688-b41f-4e26-b52b-8a3917f7cb08 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:28.489672 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09c688-b41f-4e26-b52b-8a3917f7cb08 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:28.489945 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09c688-b41f-4e26-b52b-8a3917f7cb08 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:28.490932 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b09c688-b41f-4e26-b52b-8a3917f7cb08 None admin] RBAC: Adding query filter params (name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:28.491393 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b09c688-b41f-4e26-b52b-8a3917f7cb08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:28.493224 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b09c688-b41f-4e26-b52b-8a3917f7cb08 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:28.502467 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 428/859] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 31 10:45:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:28.503627 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91783fc3-53b0-48e2-aa6b-e01769f15600 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:28.504023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91783fc3-53b0-48e2-aa6b-e01769f15600 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:28.504311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91783fc3-53b0-48e2-aa6b-e01769f15600 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:28.505683 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 429/860] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:28.511198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91783fc3-53b0-48e2-aa6b-e01769f15600 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:28.516818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70a73c-12a5-4b92-8139-62f7ebab3e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:28.517533 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70a73c-12a5-4b92-8139-62f7ebab3e0a None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:28.517808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70a73c-12a5-4b92-8139-62f7ebab3e0a None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:28.518135 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70a73c-12a5-4b92-8139-62f7ebab3e0a None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:28.518805 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70a73c-12a5-4b92-8139-62f7ebab3e0a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:28.519160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70a73c-12a5-4b92-8139-62f7ebab3e0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:28.520516 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70a73c-12a5-4b92-8139-62f7ebab3e0a None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:28.526586 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 432/861] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Aug 31 10:45:28 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:28.526730 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b9100-0e14-4f07-8e49-dc58ec361403 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:28.528247 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b9100-0e14-4f07-8e49-dc58ec361403 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:28.528531 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b9100-0e14-4f07-8e49-dc58ec361403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:28.555073 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-092b9100-0e14-4f07-8e49-dc58ec361403 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:28.556392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-092b9100-0e14-4f07-8e49-dc58ec361403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:28.578967 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:28.637335 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91783fc3-53b0-48e2-aa6b-e01769f15600 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:28.655299 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 433/862] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:45:28.680036 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c141c3e8-0644-4ee8-8977-eb97e612af2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:28.681003 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141c3e8-0644-4ee8-8977-eb97e612af2e None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:28.681455 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141c3e8-0644-4ee8-8977-eb97e612af2e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:28.681918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141c3e8-0644-4ee8-8977-eb97e612af2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:28.695572 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c141c3e8-0644-4ee8-8977-eb97e612af2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:28.697388 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c141c3e8-0644-4ee8-8977-eb97e612af2e None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:28.713549 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-c141c3e8-0644-4ee8-8977-eb97e612af2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:28.713549 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:28.713549 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:28.713549 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:28.713549 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:28.713549 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:28.713549 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:28.713549 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:28.713549 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:28.713549 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:28.713549 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:28.713549 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:28.713549 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:28.713549 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:28.713549 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:28.713549 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:28.715026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:28.715026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:28.715026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:28.715026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:28.715026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:28.715026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:28.715026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:28.715026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:28.715026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:28.715026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:28.715026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:28.715026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:28.715026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:28.715026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:28.715026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:28.715026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:28.715026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:28.715904 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:28.715904 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:28.715904 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:28.716266 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 430/863] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:28.722289 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b9100-0e14-4f07-8e49-dc58ec361403 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:28.831878 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b9100-0e14-4f07-8e49-dc58ec361403 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:28.867849 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea9eb3-0582-4036-aed0-d3d85eae8fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:28.868463 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea9eb3-0582-4036-aed0-d3d85eae8fd1 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:28.868736 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea9eb3-0582-4036-aed0-d3d85eae8fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:28.869256 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea9eb3-0582-4036-aed0-d3d85eae8fd1 None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:28.869895 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea9eb3-0582-4036-aed0-d3d85eae8fd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:28.870193 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea9eb3-0582-4036-aed0-d3d85eae8fd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:28.872158 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea9eb3-0582-4036-aed0-d3d85eae8fd1 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:28.882774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:28.887475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 434/864] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:28.893200 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c141c3e8-0644-4ee8-8977-eb97e612af2e None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:28.919946 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:28.939058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea9eb3-0582-4036-aed0-d3d85eae8fd1 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:29.016657 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c141c3e8-0644-4ee8-8977-eb97e612af2e None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:29.061825 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ba60b-6462-489a-8377-2fb5ab69742e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:29.063062 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ba60b-6462-489a-8377-2fb5ab69742e None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:29.063885 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ba60b-6462-489a-8377-2fb5ab69742e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:29.064657 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ba60b-6462-489a-8377-2fb5ab69742e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:29.066292 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea9eb3-0582-4036-aed0-d3d85eae8fd1 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:29.074725 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ba60b-6462-489a-8377-2fb5ab69742e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:29.078424 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ba60b-6462-489a-8377-2fb5ab69742e None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:29.098049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-f76ba60b-6462-489a-8377-2fb5ab69742e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 10:45:29.098049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:29.098049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:29.098049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:29.098049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:29.098049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:29.098049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:29.098049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:29.098049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:29.098049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:29.098049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:29.098049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:29.098049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:45:29.098049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:45:29.098049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:45:29.098049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:45:29.099190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:29.099190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:29.099190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:45:29.099190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:29.099190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:45:29.099190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:29.099190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:29.099190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:29.099190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:29.099190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:29.099190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:29.099190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:29.099190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:29.099190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:29.099190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:29.099190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:29.099190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:29.099190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:29.100209 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:29.100209 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:29.100209 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:45:29.100209 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:45:29.100209 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:45:29.100209 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:45:29.100209 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:45:29.100209 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:45:29.100209 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 10:45:29.100209 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:29.103258 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 431/865] 10.222.0.16 () {56 vars in 1117 bytes} [Wed Aug 31 10:45:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:29.110104 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ba60b-6462-489a-8377-2fb5ab69742e None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:29.125333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137af166-0d93-4879-a5b9-39ec550a197b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:29.127287 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137af166-0d93-4879-a5b9-39ec550a197b None admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:29.127878 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137af166-0d93-4879-a5b9-39ec550a197b None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:29.128343 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137af166-0d93-4879-a5b9-39ec550a197b None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:29.129519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137af166-0d93-4879-a5b9-39ec550a197b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:29.132015 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137af166-0d93-4879-a5b9-39ec550a197b None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:29.217214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 435/866] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Aug 31 10:45:28 2022] POST /identity/v3/services => generated 251 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 10:45:29.224954 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:29.252955 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ba60b-6462-489a-8377-2fb5ab69742e None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:29.302469 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b49b1d-284f-441b-bbf1-a73e011f58f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:29.304121 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b49b1d-284f-441b-bbf1-a73e011f58f2 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:29.304626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b49b1d-284f-441b-bbf1-a73e011f58f2 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:29.305085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b49b1d-284f-441b-bbf1-a73e011f58f2 None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:29.306059 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b49b1d-284f-441b-bbf1-a73e011f58f2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:29.306603 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b49b1d-284f-441b-bbf1-a73e011f58f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:29.308407 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b49b1d-284f-441b-bbf1-a73e011f58f2 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:29.329675 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 432/867] 10.222.0.16 () {56 vars in 1149 bytes} [Wed Aug 31 10:45:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:29.335874 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137af166-0d93-4879-a5b9-39ec550a197b None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:29.408608 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:29.453215 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137af166-0d93-4879-a5b9-39ec550a197b None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:29.489710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730eaf6-483e-4b1d-b33e-4fd700418707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:29.490315 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730eaf6-483e-4b1d-b33e-4fd700418707 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:29.490618 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730eaf6-483e-4b1d-b33e-4fd700418707 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:29.490889 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730eaf6-483e-4b1d-b33e-4fd700418707 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:29.563493 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730eaf6-483e-4b1d-b33e-4fd700418707 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:29.564877 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730eaf6-483e-4b1d-b33e-4fd700418707 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:29.629333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:29.674970 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 436/868] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:45:29 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:45:29.680676 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b49b1d-284f-441b-bbf1-a73e011f58f2 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:29.790100 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b49b1d-284f-441b-bbf1-a73e011f58f2 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:29.822853 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:29.827874 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f577a-151d-4081-aa1a-2bcc62f86809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:29.828520 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f577a-151d-4081-aa1a-2bcc62f86809 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:29.828792 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f577a-151d-4081-aa1a-2bcc62f86809 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:29.829059 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f577a-151d-4081-aa1a-2bcc62f86809 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:29.840243 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f577a-151d-4081-aa1a-2bcc62f86809 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:29.841711 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f577a-151d-4081-aa1a-2bcc62f86809 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:29.857486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-e03f577a-151d-4081-aa1a-2bcc62f86809 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:29.857486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:29.857486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:29.857486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:29.857486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:29.857486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:29.857486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:29.857486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:29.857486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:29.857486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:29.857486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:29.857486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:29.857486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:29.857486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:29.857486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:29.857486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:29.858513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:29.858513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:29.858513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:29.858513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:29.858513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:29.858513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:29.858513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:29.858513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:29.858513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:29.858513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:29.858513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:29.858513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:29.858513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:29.858513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:29.858513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:29.858513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:29.858513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:29.859373 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:29.859373 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:29.859373 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:29.859373 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 433/869] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:29.864611 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730eaf6-483e-4b1d-b33e-4fd700418707 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:29.944130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef730b28-eaba-475d-a159-e8458c1ee453 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:29.944469 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef730b28-eaba-475d-a159-e8458c1ee453 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:29.944679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef730b28-eaba-475d-a159-e8458c1ee453 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:29.946371 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 434/870] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:29.952529 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdbb9c-7d9d-4864-872a-ccb2569259d1 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:29.952839 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdbb9c-7d9d-4864-872a-ccb2569259d1 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:29.953191 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdbb9c-7d9d-4864-872a-ccb2569259d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:29.972440 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730eaf6-483e-4b1d-b33e-4fd700418707 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:29.976669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-6fcdbb9c-7d9d-4864-872a-ccb2569259d1 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:29.978102 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-6fcdbb9c-7d9d-4864-872a-ccb2569259d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:30.018041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60574340-3c99-4ca9-afac-216fbaac2ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:30.019014 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60574340-3c99-4ca9-afac-216fbaac2ada None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:30.019538 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60574340-3c99-4ca9-afac-216fbaac2ada None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:30.020008 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60574340-3c99-4ca9-afac-216fbaac2ada None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:30.020836 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60574340-3c99-4ca9-afac-216fbaac2ada None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:30.021330 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60574340-3c99-4ca9-afac-216fbaac2ada None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:30.022989 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60574340-3c99-4ca9-afac-216fbaac2ada None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:30.036233 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 437/871] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:30.042443 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60574340-3c99-4ca9-afac-216fbaac2ada None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:30.065977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 435/872] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:29 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:45:30.082860 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:30.168374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60574340-3c99-4ca9-afac-216fbaac2ada None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:30.204830 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187404e-2514-42a7-b4c5-fbb8c7962ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:30.205460 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187404e-2514-42a7-b4c5-fbb8c7962ad1 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:30.205738 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187404e-2514-42a7-b4c5-fbb8c7962ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:30.206013 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187404e-2514-42a7-b4c5-fbb8c7962ad1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:30.213294 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e187404e-2514-42a7-b4c5-fbb8c7962ad1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:30.214816 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e187404e-2514-42a7-b4c5-fbb8c7962ad1 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:30.226790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-e187404e-2514-42a7-b4c5-fbb8c7962ad1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:30.226790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:30.226790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:30.226790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:30.226790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:30.226790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:30.226790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:30.226790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:30.226790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:30.226790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:30.226790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:30.226790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:30.226790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:45:30.226790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:45:30.226790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:45:30.227737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:45:30.227737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:30.227737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:30.227737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:30.227737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:30.227737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:30.227737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:30.227737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:30.227737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:30.227737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:30.227737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:30.227737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:30.227737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:30.227737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:30.227737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:30.227737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:30.227737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:30.228642 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:45:30.228642 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:45:30.228642 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:45:30.228642 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:45:30.228642 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:45:30.228642 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:45:30.228642 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:30.228642 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:30.228642 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 438/873] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:45:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:30.234901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcdbb9c-7d9d-4864-872a-ccb2569259d1 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:30.349275 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcdbb9c-7d9d-4864-872a-ccb2569259d1 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:30.365111 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:30.375423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187404e-2514-42a7-b4c5-fbb8c7962ad1 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:30.391049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c4a59-aa9d-4a5f-851f-c103f17f4c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:30.391833 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c4a59-aa9d-4a5f-851f-c103f17f4c0d None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:30.392115 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c4a59-aa9d-4a5f-851f-c103f17f4c0d None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:30.392442 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c4a59-aa9d-4a5f-851f-c103f17f4c0d None admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:30.393293 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c4a59-aa9d-4a5f-851f-c103f17f4c0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:30.394016 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c4a59-aa9d-4a5f-851f-c103f17f4c0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:30.396689 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c4a59-aa9d-4a5f-851f-c103f17f4c0d None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:30.414000 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 436/874] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:45:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:30.420159 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c4a59-aa9d-4a5f-851f-c103f17f4c0d None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:30.472388 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:30.496799 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187404e-2514-42a7-b4c5-fbb8c7962ad1 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:30.531662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c4a59-aa9d-4a5f-851f-c103f17f4c0d None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:30.536074 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b432b-f914-4d54-a960-fa2f66254465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:30.537405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b432b-f914-4d54-a960-fa2f66254465 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:30.538062 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b432b-f914-4d54-a960-fa2f66254465 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:30.538681 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b432b-f914-4d54-a960-fa2f66254465 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:30.539531 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b432b-f914-4d54-a960-fa2f66254465 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:30.541204 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b432b-f914-4d54-a960-fa2f66254465 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:30.551728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-ce1b432b-f914-4d54-a960-fa2f66254465 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:45:30.551728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:30.551728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:45:30.551728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:45:30.551728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:45:30.551728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:45:30.551728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:45:30.551728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:30.551728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:45:30.551728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:30.551728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:45:30.551728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:30.551728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:30.551728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:30.551728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:30.551728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:30.551728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:30.552801 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:30.552801 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:30.552801 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:30.552801 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:30.552801 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:30.552801 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:30.552801 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:45:30.552801 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:45:30.552801 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:45:30.552801 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:45:30.552801 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:30.552801 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:30.552801 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:30.552801 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:30.552801 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:30.552801 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:30.553636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:30.553636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:30.553636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:30.553636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:30.553636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:30.553636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:30.553636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:30.553636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:30.553636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:30.553636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:30.553636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:45:30.553636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:30.553636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:45:30.553636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:30.562218 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 439/875] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Aug 31 10:45:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:30.563582 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b432b-f914-4d54-a960-fa2f66254465 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:30.576176 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb18405-723e-44f3-abf6-d0169f86a8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:30.577272 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb18405-723e-44f3-abf6-d0169f86a8f5 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:30.577971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb18405-723e-44f3-abf6-d0169f86a8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:30.578233 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb18405-723e-44f3-abf6-d0169f86a8f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:30.579682 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb18405-723e-44f3-abf6-d0169f86a8f5 None admin] RBAC: Adding query filter params (role.id=bec755d6565a41798b09f5f57c1d51dd, scope.project.id=8db9b3f8a9f041938ac7b39003a35015, user.id=7796ebff70054de090e02c58bf7000f4) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:30.580296 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb18405-723e-44f3-abf6-d0169f86a8f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:30.583131 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb18405-723e-44f3-abf6-d0169f86a8f5 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:30.599903 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 437/876] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:45:30 2022] GET /identity/v3/role_assignments?user.id=7796ebff70054de090e02c58bf7000f4&scope.project.id=8db9b3f8a9f041938ac7b39003a35015&role.id=bec755d6565a41798b09f5f57c1d51dd => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:30.677435 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b432b-f914-4d54-a960-fa2f66254465 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:30.680170 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32cea2-f177-47b6-88d2-7233ffdeb629 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:30.680471 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32cea2-f177-47b6-88d2-7233ffdeb629 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:30.680727 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32cea2-f177-47b6-88d2-7233ffdeb629 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:30.682574 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 438/877] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:45:30.691290 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4614d-8fa7-430e-aa51-7ac54a72096e None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:30.691628 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4614d-8fa7-430e-aa51-7ac54a72096e None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:30.691907 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4614d-8fa7-430e-aa51-7ac54a72096e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:30.712194 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-8db4614d-8fa7-430e-aa51-7ac54a72096e None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:30.713850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-8db4614d-8fa7-430e-aa51-7ac54a72096e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:30.723372 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1562631d-dde3-4903-8a32-f85bfaf46386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:30.724391 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562631d-dde3-4903-8a32-f85bfaf46386 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:30.724979 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562631d-dde3-4903-8a32-f85bfaf46386 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:30.725466 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562631d-dde3-4903-8a32-f85bfaf46386 None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:30.726444 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1562631d-dde3-4903-8a32-f85bfaf46386 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:30.726952 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1562631d-dde3-4903-8a32-f85bfaf46386 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:30.728572 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1562631d-dde3-4903-8a32-f85bfaf46386 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:30.740352 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 440/878] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Aug 31 10:45:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:30.742008 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1562631d-dde3-4903-8a32-f85bfaf46386 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:30.754489 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:30.760876 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:30.823533 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 439/879] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:30 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:45:30.885251 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1562631d-dde3-4903-8a32-f85bfaf46386 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:30.922279 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d56bd-b14a-4365-b2bf-d7095785980c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:30.923262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d56bd-b14a-4365-b2bf-d7095785980c None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:30.923714 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d56bd-b14a-4365-b2bf-d7095785980c None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:30.924167 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d56bd-b14a-4365-b2bf-d7095785980c None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:30.924968 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98d56bd-b14a-4365-b2bf-d7095785980c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:30.925450 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98d56bd-b14a-4365-b2bf-d7095785980c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:30.927017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98d56bd-b14a-4365-b2bf-d7095785980c None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:30.932342 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 441/880] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Aug 31 10:45:30 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:31.099381 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4614d-8fa7-430e-aa51-7ac54a72096e None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:31.209876 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4614d-8fa7-430e-aa51-7ac54a72096e None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:31.247194 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c61dd-3e8c-47d1-9439-9ca108263f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:31.247801 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c61dd-3e8c-47d1-9439-9ca108263f15 None admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:31.248073 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c61dd-3e8c-47d1-9439-9ca108263f15 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:31.248337 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c61dd-3e8c-47d1-9439-9ca108263f15 None admin] PATH_INFO: `/v3/services` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:31.249002 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c61dd-3e8c-47d1-9439-9ca108263f15 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:31.250358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c61dd-3e8c-47d1-9439-9ca108263f15 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:31.345261 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7580d4-9354-40e4-95ce-15a141959718 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:31.345840 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7580d4-9354-40e4-95ce-15a141959718 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:31.345840 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7580d4-9354-40e4-95ce-15a141959718 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:31.347393 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 442/881] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:45:31.352446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326a229-4079-4d42-9fd8-c375f39d8125 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:31.352806 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326a229-4079-4d42-9fd8-c375f39d8125 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:31.352992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326a229-4079-4d42-9fd8-c375f39d8125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:31.373057 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-e326a229-4079-4d42-9fd8-c375f39d8125 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:31.373442 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 440/882] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Aug 31 10:45:31 2022] POST /identity/v3/services => generated 269 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:45:31.374242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-e326a229-4079-4d42-9fd8-c375f39d8125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:31.455304 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:31.462163 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 443/883] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:31 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:45:31.741449 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c61dd-3e8c-47d1-9439-9ca108263f15 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:31.773516 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:31.854471 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c61dd-3e8c-47d1-9439-9ca108263f15 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:31.894917 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b30970-5617-4cc1-a4b1-c8f03559c8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:31.895811 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b30970-5617-4cc1-a4b1-c8f03559c8ab None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:31.896271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b30970-5617-4cc1-a4b1-c8f03559c8ab None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:31.896703 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b30970-5617-4cc1-a4b1-c8f03559c8ab None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:31.897437 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b30970-5617-4cc1-a4b1-c8f03559c8ab None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:31.898991 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b30970-5617-4cc1-a4b1-c8f03559c8ab None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:31.908745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-d1b30970-5617-4cc1-a4b1-c8f03559c8ab None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 10:45:31.908745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:31.908745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:45:31.908745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:45:31.908745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:45:31.908745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:45:31.908745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:45:31.908745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:31.908745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 10:45:31.908745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:31.908745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:45:31.908745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:31.908745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:31.908745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:31.908745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:31.908745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:31.908745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:31.909964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:31.909964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:31.909964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:31.909964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:31.909964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:31.909964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:31.909964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:45:31.909964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:45:31.909964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:45:31.909964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:45:31.909964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:31.909964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:31.909964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:31.909964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:31.909964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:31.909964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:31.911023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:31.911023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:31.911023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:31.911023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:31.911023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:31.911023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:31.911023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:31.911023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:31.911023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:31.911023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:31.911023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:45:31.911023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:31.911023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 10:45:31.911023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:31.913432 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 441/884] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 31 10:45:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:31.919937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326a229-4079-4d42-9fd8-c375f39d8125 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:32.025229 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326a229-4079-4d42-9fd8-c375f39d8125 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:32.061290 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae3fbec-14b5-498f-bc89-5389c36736f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:32.061906 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3fbec-14b5-498f-bc89-5389c36736f8 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:32.062179 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3fbec-14b5-498f-bc89-5389c36736f8 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:32.062473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3fbec-14b5-498f-bc89-5389c36736f8 None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:32.063107 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae3fbec-14b5-498f-bc89-5389c36736f8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:32.063404 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae3fbec-14b5-498f-bc89-5389c36736f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:32.064763 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae3fbec-14b5-498f-bc89-5389c36736f8 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:32.069982 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 444/885] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 31 10:45:31 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:32.076008 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b30970-5617-4cc1-a4b1-c8f03559c8ab None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:32.179135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:32.189091 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b30970-5617-4cc1-a4b1-c8f03559c8ab None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:32.226334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac02f854-f810-4993-a06a-500d2d6ec713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:32.226975 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02f854-f810-4993-a06a-500d2d6ec713 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:32.227255 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02f854-f810-4993-a06a-500d2d6ec713 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:32.227522 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02f854-f810-4993-a06a-500d2d6ec713 None admin] PATH_INFO: `/v3/services` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:32.228157 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac02f854-f810-4993-a06a-500d2d6ec713 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:32.228456 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac02f854-f810-4993-a06a-500d2d6ec713 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:32.230434 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac02f854-f810-4993-a06a-500d2d6ec713 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:32.240666 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 442/886] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 31 10:45:32 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:32.242090 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae3fbec-14b5-498f-bc89-5389c36736f8 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:32.319833 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:32.362039 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae3fbec-14b5-498f-bc89-5389c36736f8 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:32.399197 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec8bf6-af25-4c92-98bf-246cd109d5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:32.399855 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec8bf6-af25-4c92-98bf-246cd109d5a5 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:32.400137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec8bf6-af25-4c92-98bf-246cd109d5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:32.400407 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec8bf6-af25-4c92-98bf-246cd109d5a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:32.401119 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec8bf6-af25-4c92-98bf-246cd109d5a5 None admin] RBAC: Adding query filter params (interface=public, service_id=6267b96d330e4b109c22baaf6c524f2c, region_id=RegionOne) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:32.401425 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec8bf6-af25-4c92-98bf-246cd109d5a5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:32.403725 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec8bf6-af25-4c92-98bf-246cd109d5a5 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:32.409799 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 445/887] 10.222.0.16 () {56 vars in 1251 bytes} [Wed Aug 31 10:45:32 2022] GET /identity/v3/endpoints?service_id=6267b96d330e4b109c22baaf6c524f2c&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:32.415499 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac02f854-f810-4993-a06a-500d2d6ec713 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:32.532765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac02f854-f810-4993-a06a-500d2d6ec713 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:32.568623 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365c964-90a5-4c93-9026-c59969c476ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:32.569645 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365c964-90a5-4c93-9026-c59969c476ce None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:32.570183 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365c964-90a5-4c93-9026-c59969c476ce None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:32.570766 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365c964-90a5-4c93-9026-c59969c476ce None admin] PATH_INFO: `/v3/services` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:32.571588 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365c964-90a5-4c93-9026-c59969c476ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:32.573241 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365c964-90a5-4c93-9026-c59969c476ce None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:32.586621 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 443/888] 10.222.0.16 () {56 vars in 1087 bytes} [Wed Aug 31 10:45:32 2022] GET /identity/v3/services => generated 1268 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:45:32.770557 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab99853-c329-4bd9-8e2b-48ed8dd3c4e9 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:32.770925 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab99853-c329-4bd9-8e2b-48ed8dd3c4e9 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:32.771107 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab99853-c329-4bd9-8e2b-48ed8dd3c4e9 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:32.772700 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 446/889] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:32.778032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb2001-de3c-4764-8b5a-306795e75079 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:32.778377 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb2001-de3c-4764-8b5a-306795e75079 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:32.778674 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb2001-de3c-4764-8b5a-306795e75079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:32.802342 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-40bb2001-de3c-4764-8b5a-306795e75079 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:32.803578 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-40bb2001-de3c-4764-8b5a-306795e75079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:32.819754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:32.882808 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 444/890] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:32 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:45:32.976118 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ad4c7-81fc-47d7-93b4-aede22329c90 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:32.976439 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ad4c7-81fc-47d7-93b4-aede22329c90 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:32.976681 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ad4c7-81fc-47d7-93b4-aede22329c90 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:32.978281 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 447/891] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:32.984239 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c727742-a6fc-4919-ba4f-2451afa63387 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:32.984774 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c727742-a6fc-4919-ba4f-2451afa63387 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:32.985260 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c727742-a6fc-4919-ba4f-2451afa63387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:33.004759 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-2c727742-a6fc-4919-ba4f-2451afa63387 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:33.006341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-2c727742-a6fc-4919-ba4f-2451afa63387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:33.084082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 445/892] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:32 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:45:33.173057 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ad4c7-81fc-47d7-93b4-aede22329c90 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:33.281026 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ad4c7-81fc-47d7-93b4-aede22329c90 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:33.310275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:33.320248 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7d403-84e8-490c-b7fb-2b0533070980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:33.320889 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7d403-84e8-490c-b7fb-2b0533070980 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:33.321277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7d403-84e8-490c-b7fb-2b0533070980 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:33.321552 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7d403-84e8-490c-b7fb-2b0533070980 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:33.334228 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f7d403-84e8-490c-b7fb-2b0533070980 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:45:33.340259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-c1f7d403-84e8-490c-b7fb-2b0533070980 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 10:45:33.340259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:33.340259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:33.340259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:33.340259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:33.340259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:33.340259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:33.340259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:33.340259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:33.340259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:33.340259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:33.340259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:33.340259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:45:33.340259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:45:33.340259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:45:33.340259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:45:33.341241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:45:33.341241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:45:33.341241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:45:33.341241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:45:33.341241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:33.341241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:33.341241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:33.341241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:33.341241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:33.341241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:33.341241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:33.341241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:33.341241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:33.341241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:33.341241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:33.341241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:33.342081 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:33.342081 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:33.342081 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:33.342081 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:33.342081 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:45:33.342081 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:45:33.342081 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:45:33.342081 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:45:33.342081 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:45:33.342081 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:45:33.342081 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 10:45:33.342081 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:33.342081 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 448/893] 10.222.0.16 () {56 vars in 1120 bytes} [Wed Aug 31 10:45:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:33.347933 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c727742-a6fc-4919-ba4f-2451afa63387 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:33.385630 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7d403-84e8-490c-b7fb-2b0533070980 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:33.447239 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:33.468311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c727742-a6fc-4919-ba4f-2451afa63387 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:33.502989 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fc39b-f3cd-459b-afbc-9da18756ea73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:33.503591 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fc39b-f3cd-459b-afbc-9da18756ea73 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:33.503866 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fc39b-f3cd-459b-afbc-9da18756ea73 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:33.504129 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fc39b-f3cd-459b-afbc-9da18756ea73 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:33.504786 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1fc39b-f3cd-459b-afbc-9da18756ea73 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:33.505083 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1fc39b-f3cd-459b-afbc-9da18756ea73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:33.506448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1fc39b-f3cd-459b-afbc-9da18756ea73 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:33.510700 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7d403-84e8-490c-b7fb-2b0533070980 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:33.519270 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 446/894] 10.222.0.16 () {56 vars in 1115 bytes} [Wed Aug 31 10:45:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:33.519270 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e3564-7a93-4014-83fd-9920c03990fb None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:33.519270 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e3564-7a93-4014-83fd-9920c03990fb None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:33.519270 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e3564-7a93-4014-83fd-9920c03990fb None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:33.519270 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 447/895] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:33.520207 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e3564-7a93-4014-83fd-9920c03990fb None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:33.547903 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d5b24-8d4f-4c6b-bd93-f09b241e23f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:33.548506 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d5b24-8d4f-4c6b-bd93-f09b241e23f3 None admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:33.548777 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d5b24-8d4f-4c6b-bd93-f09b241e23f3 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:33.549043 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d5b24-8d4f-4c6b-bd93-f09b241e23f3 None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:33.549689 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1d5b24-8d4f-4c6b-bd93-f09b241e23f3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:33.551075 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1d5b24-8d4f-4c6b-bd93-f09b241e23f3 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:33.614786 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 449/896] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Aug 31 10:45:33 2022] POST /identity/v3/services => generated 241 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 10:45:33.616141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c663f20-b3ed-46ba-808d-de0e1237d033 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:33.616453 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c663f20-b3ed-46ba-808d-de0e1237d033 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:33.616709 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c663f20-b3ed-46ba-808d-de0e1237d033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:33.638420 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-9c663f20-b3ed-46ba-808d-de0e1237d033 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:33.640008 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-9c663f20-b3ed-46ba-808d-de0e1237d033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:33.646671 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e3564-7a93-4014-83fd-9920c03990fb None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:33.697954 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:33.707439 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545375b-38df-406b-a823-b8071d17c90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:33.710887 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545375b-38df-406b-a823-b8071d17c90e None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:33.710887 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545375b-38df-406b-a823-b8071d17c90e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:33.710887 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545375b-38df-406b-a823-b8071d17c90e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:33.720298 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:33.726760 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545375b-38df-406b-a823-b8071d17c90e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:33.728963 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545375b-38df-406b-a823-b8071d17c90e None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:33.733002 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 450/897] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:45:33.748333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-e545375b-38df-406b-a823-b8071d17c90e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:33.748333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:33.748333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:33.748333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:33.748333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:33.748333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:33.748333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:33.748333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:33.748333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:33.748333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:33.748333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:33.748333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:33.748333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:33.748333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:33.748333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:33.748333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:33.749334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:33.749334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:33.749334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:33.749334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:33.749334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:33.749334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:33.749334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:33.749334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:33.749334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:33.749334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:33.749334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:33.749334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:33.749334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:33.749334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:33.749334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:33.749334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:33.749334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:33.750224 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:33.750224 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:33.750224 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:33.752689 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 448/898] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:33.762423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c663f20-b3ed-46ba-808d-de0e1237d033 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:33.875553 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c663f20-b3ed-46ba-808d-de0e1237d033 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:33.911610 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e42934-4010-46cb-b6fe-0e9ef71df935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:33.912216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e42934-4010-46cb-b6fe-0e9ef71df935 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:33.912528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e42934-4010-46cb-b6fe-0e9ef71df935 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:33.912759 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e42934-4010-46cb-b6fe-0e9ef71df935 None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:33.913455 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e42934-4010-46cb-b6fe-0e9ef71df935 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:33.913759 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e42934-4010-46cb-b6fe-0e9ef71df935 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:33.915191 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e42934-4010-46cb-b6fe-0e9ef71df935 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:33.927273 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 451/899] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:33.932627 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545375b-38df-406b-a823-b8071d17c90e None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:34.036155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e42934-4010-46cb-b6fe-0e9ef71df935 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:34.044917 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545375b-38df-406b-a823-b8071d17c90e None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:34.085290 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:34.091677 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf16c49-8a68-4fdf-bfcc-8bd58766ab66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:34.097102 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf16c49-8a68-4fdf-bfcc-8bd58766ab66 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:34.097102 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf16c49-8a68-4fdf-bfcc-8bd58766ab66 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:34.097102 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf16c49-8a68-4fdf-bfcc-8bd58766ab66 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:34.112881 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf16c49-8a68-4fdf-bfcc-8bd58766ab66 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:34.114374 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf16c49-8a68-4fdf-bfcc-8bd58766ab66 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:34.127354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-aaf16c49-8a68-4fdf-bfcc-8bd58766ab66 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 10:45:34.127354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:34.127354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:34.127354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:34.127354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:34.127354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:34.127354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:34.127354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:34.127354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:34.127354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:34.127354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:34.127354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:34.127354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:45:34.127354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:45:34.127354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:45:34.127354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:45:34.128328 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:34.128328 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:34.128328 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:45:34.128328 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:34.128328 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:45:34.128328 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:34.128328 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:34.128328 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:34.128328 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:34.128328 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:34.128328 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:34.128328 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:34.128328 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:34.128328 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:34.128328 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:34.128328 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:34.128328 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:34.128328 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:34.129253 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:34.129253 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:34.129253 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:45:34.129253 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:45:34.129253 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:45:34.129253 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:45:34.129253 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:45:34.129253 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:45:34.129253 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 10:45:34.129253 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:34.130600 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 449/900] 10.222.0.16 () {56 vars in 1117 bytes} [Wed Aug 31 10:45:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:34.139487 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf16c49-8a68-4fdf-bfcc-8bd58766ab66 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:34.182707 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e42934-4010-46cb-b6fe-0e9ef71df935 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:34.206715 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:34.220279 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a6770-b66d-4891-8045-3688717f75ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:34.221059 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a6770-b66d-4891-8045-3688717f75ee None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:34.221405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a6770-b66d-4891-8045-3688717f75ee None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:34.221684 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a6770-b66d-4891-8045-3688717f75ee None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:34.222279 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a6770-b66d-4891-8045-3688717f75ee None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:34.223681 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a6770-b66d-4891-8045-3688717f75ee None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:34.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-c49a6770-b66d-4891-8045-3688717f75ee None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 10:45:34.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:34.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:45:34.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:45:34.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:45:34.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:45:34.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:45:34.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:34.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 10:45:34.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:34.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:45:34.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:34.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:34.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:34.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:34.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:34.235270 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:34.235270 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:34.235270 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:34.235270 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:34.235270 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:34.235270 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:34.235270 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:34.235270 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:45:34.235270 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:45:34.235270 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:45:34.235270 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:45:34.235270 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:34.235270 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:34.235270 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:34.235270 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:34.235270 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:34.235270 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:34.241695 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:34.241695 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:34.241695 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:34.241695 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:34.241695 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:34.241695 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:34.241695 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:34.241695 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:34.241695 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:34.241695 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:34.241695 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:45:34.241695 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:34.241695 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 10:45:34.241695 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:34.241695 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 452/901] 10.222.0.16 () {56 vars in 1132 bytes} [Wed Aug 31 10:45:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:34.242528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a6770-b66d-4891-8045-3688717f75ee None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:34.281131 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf16c49-8a68-4fdf-bfcc-8bd58766ab66 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:34.332520 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63e4d6-b1ae-438b-94fa-73f9e338910d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:34.333035 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63e4d6-b1ae-438b-94fa-73f9e338910d None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:34.333301 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63e4d6-b1ae-438b-94fa-73f9e338910d None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:34.333568 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63e4d6-b1ae-438b-94fa-73f9e338910d None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:34.334314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc63e4d6-b1ae-438b-94fa-73f9e338910d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:34.334641 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc63e4d6-b1ae-438b-94fa-73f9e338910d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:34.336475 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc63e4d6-b1ae-438b-94fa-73f9e338910d None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:34.362441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 450/902] 10.222.0.16 () {56 vars in 1149 bytes} [Wed Aug 31 10:45:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:34.369570 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63e4d6-b1ae-438b-94fa-73f9e338910d None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:34.389494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:34.395331 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a6770-b66d-4891-8045-3688717f75ee None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:34.441366 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f629d7-1122-4b10-9359-266b7692ac16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:34.442129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f629d7-1122-4b10-9359-266b7692ac16 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:34.442476 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f629d7-1122-4b10-9359-266b7692ac16 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:34.442826 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f629d7-1122-4b10-9359-266b7692ac16 None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:34.443941 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f629d7-1122-4b10-9359-266b7692ac16 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:34.443941 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f629d7-1122-4b10-9359-266b7692ac16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:34.445755 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f629d7-1122-4b10-9359-266b7692ac16 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:34.456780 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 453/903] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Aug 31 10:45:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:34.457886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f629d7-1122-4b10-9359-266b7692ac16 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:34.516464 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63e4d6-b1ae-438b-94fa-73f9e338910d None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:34.528640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:34.560705 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce03a18-664b-4c62-b4f5-8915e8397df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:34.561915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce03a18-664b-4c62-b4f5-8915e8397df1 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:34.562323 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce03a18-664b-4c62-b4f5-8915e8397df1 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:34.562702 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce03a18-664b-4c62-b4f5-8915e8397df1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:34.600379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f629d7-1122-4b10-9359-266b7692ac16 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:34.643966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c2b3a-8aab-403e-b498-5dc3766a362a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:34.644592 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c2b3a-8aab-403e-b498-5dc3766a362a None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:34.644863 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c2b3a-8aab-403e-b498-5dc3766a362a None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:34.645139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c2b3a-8aab-403e-b498-5dc3766a362a None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:34.645776 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c2b3a-8aab-403e-b498-5dc3766a362a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:34.646075 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c2b3a-8aab-403e-b498-5dc3766a362a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:34.647535 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c2b3a-8aab-403e-b498-5dc3766a362a None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:34.648034 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce03a18-664b-4c62-b4f5-8915e8397df1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:34.650375 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce03a18-664b-4c62-b4f5-8915e8397df1 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:34.653412 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 454/904] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Aug 31 10:45:34 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:34.659747 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024f611-23d5-4490-9e33-8d515e507c8e None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:34.659747 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024f611-23d5-4490-9e33-8d515e507c8e None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:34.659747 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024f611-23d5-4490-9e33-8d515e507c8e None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:34.659747 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 455/905] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:34.661207 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024f611-23d5-4490-9e33-8d515e507c8e None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:34.691146 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:34.775462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024f611-23d5-4490-9e33-8d515e507c8e None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:34.780836 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 451/906] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:45:34 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:45:34.782137 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428cdae0-4569-43c5-953c-e50c24b62ff4 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:34.782827 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428cdae0-4569-43c5-953c-e50c24b62ff4 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:34.782827 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428cdae0-4569-43c5-953c-e50c24b62ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:34.802331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-428cdae0-4569-43c5-953c-e50c24b62ff4 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:34.803596 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-428cdae0-4569-43c5-953c-e50c24b62ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:34.805787 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:34.816035 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7fa23a-13d6-4822-84d2-50ac816c1a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:34.816644 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7fa23a-13d6-4822-84d2-50ac816c1a0b None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:34.816921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7fa23a-13d6-4822-84d2-50ac816c1a0b None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:34.817189 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7fa23a-13d6-4822-84d2-50ac816c1a0b None admin] PATH_INFO: `/v3/endpoints` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:34.817895 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7fa23a-13d6-4822-84d2-50ac816c1a0b None admin] RBAC: Adding query filter params (interface=public, service_id=09a13b35d8114d199a35309466475054, region_id=RegionOne) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:34.818196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7fa23a-13d6-4822-84d2-50ac816c1a0b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:34.819606 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7fa23a-13d6-4822-84d2-50ac816c1a0b None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:34.825169 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 456/907] 10.222.0.16 () {56 vars in 1251 bytes} [Wed Aug 31 10:45:34 2022] GET /identity/v3/endpoints?service_id=09a13b35d8114d199a35309466475054&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:34.826022 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7fa23a-13d6-4822-84d2-50ac816c1a0b None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:34.891485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 452/908] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:45:34.892254 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428cdae0-4569-43c5-953c-e50c24b62ff4 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:34.949761 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:34.961634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7fa23a-13d6-4822-84d2-50ac816c1a0b None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:35.001073 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b4ae9-8c98-4c46-a3e3-a301176999b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:35.001737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b4ae9-8c98-4c46-a3e3-a301176999b8 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:35.002013 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b4ae9-8c98-4c46-a3e3-a301176999b8 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:35.002272 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b4ae9-8c98-4c46-a3e3-a301176999b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:35.013565 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b4ae9-8c98-4c46-a3e3-a301176999b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:35.015080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b4ae9-8c98-4c46-a3e3-a301176999b8 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:35.023561 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428cdae0-4569-43c5-953c-e50c24b62ff4 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:35.030774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-bf3b4ae9-8c98-4c46-a3e3-a301176999b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:35.030774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:35.030774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:35.030774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:35.030774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:35.030774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:35.030774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:35.030774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:35.030774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:35.030774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:35.030774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:35.030774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:35.030774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:35.030774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:35.030774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:35.030774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:35.031891 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:35.031891 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:35.031891 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:35.031891 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:35.031891 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:35.031891 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:35.031891 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:35.031891 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:35.031891 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:35.031891 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:35.031891 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:35.031891 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:35.031891 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:35.031891 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:35.031891 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:35.031891 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:35.031891 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:35.032796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:35.032796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:35.032796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:35.036780 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 457/909] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:35.037712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b4ae9-8c98-4c46-a3e3-a301176999b8 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:35.069377 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8bd95f-7ee5-40c6-a55e-b7a48c98b780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:35.070368 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8bd95f-7ee5-40c6-a55e-b7a48c98b780 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:35.070931 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8bd95f-7ee5-40c6-a55e-b7a48c98b780 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:35.071513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8bd95f-7ee5-40c6-a55e-b7a48c98b780 None admin] PATH_INFO: `/v3/services` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:35.072458 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8bd95f-7ee5-40c6-a55e-b7a48c98b780 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:35.074350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8bd95f-7ee5-40c6-a55e-b7a48c98b780 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:35.082280 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 453/910] 10.222.0.16 () {56 vars in 1087 bytes} [Wed Aug 31 10:45:34 2022] GET /identity/v3/services => generated 1497 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:45:35.163648 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b4ae9-8c98-4c46-a3e3-a301176999b8 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:35.186343 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8bd95f-7ee5-40c6-a55e-b7a48c98b780 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:35.189828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:35.206197 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d7089-bbec-41a1-acc8-fd0adbef6f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:35.206874 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d7089-bbec-41a1-acc8-fd0adbef6f1d None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:35.207181 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d7089-bbec-41a1-acc8-fd0adbef6f1d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:35.207449 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d7089-bbec-41a1-acc8-fd0adbef6f1d None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:35.208364 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7d7089-bbec-41a1-acc8-fd0adbef6f1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:35.208669 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7d7089-bbec-41a1-acc8-fd0adbef6f1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:35.210082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7d7089-bbec-41a1-acc8-fd0adbef6f1d None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:35.224525 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 458/911] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:35.230703 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d7089-bbec-41a1-acc8-fd0adbef6f1d None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:35.236236 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:35.341928 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8bd95f-7ee5-40c6-a55e-b7a48c98b780 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:35.374886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d7089-bbec-41a1-acc8-fd0adbef6f1d None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:35.379501 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eff682-47e3-40f3-b2b4-57252af10eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:35.380143 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eff682-47e3-40f3-b2b4-57252af10eac None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:35.380414 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eff682-47e3-40f3-b2b4-57252af10eac None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:35.380681 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eff682-47e3-40f3-b2b4-57252af10eac None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:35.381259 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eff682-47e3-40f3-b2b4-57252af10eac None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:35.382643 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eff682-47e3-40f3-b2b4-57252af10eac None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:35.392598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-15eff682-47e3-40f3-b2b4-57252af10eac None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 10:45:35.392598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:35.392598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:45:35.392598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:45:35.392598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:45:35.392598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:45:35.392598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:45:35.392598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:35.392598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 10:45:35.392598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:35.392598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:45:35.392598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:35.392598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:35.392598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:35.392598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:35.392598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:35.392598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:35.393712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:35.393712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:35.393712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:35.393712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:35.393712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:35.393712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:35.393712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:45:35.393712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:45:35.393712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:45:35.393712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:45:35.393712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:35.393712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:35.393712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:35.393712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:35.393712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:35.393712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:35.394997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:35.394997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:35.394997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:35.394997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:35.394997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:35.394997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:35.394997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:35.394997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:35.394997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:35.394997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:35.394997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:45:35.394997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:35.394997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 10:45:35.394997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:35.394997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 454/912] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 31 10:45:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:35.403108 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eff682-47e3-40f3-b2b4-57252af10eac None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:35.418291 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfeeaa0-dc3f-4126-92dd-4a57d438e23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:35.419656 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfeeaa0-dc3f-4126-92dd-4a57d438e23f None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:35.420158 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfeeaa0-dc3f-4126-92dd-4a57d438e23f None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:35.420628 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfeeaa0-dc3f-4126-92dd-4a57d438e23f None admin] PATH_INFO: `/v3/projects/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:35.428501 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfeeaa0-dc3f-4126-92dd-4a57d438e23f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:35.430265 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfeeaa0-dc3f-4126-92dd-4a57d438e23f None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:35.443214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-ccfeeaa0-dc3f-4126-92dd-4a57d438e23f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:35.443214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:35.443214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:35.443214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:35.443214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:35.443214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:35.443214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:35.443214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:35.443214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:35.443214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:35.443214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:35.443214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:35.443214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:45:35.443214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:45:35.443214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:45:35.444155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:45:35.444155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:35.444155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:35.444155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:35.444155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:35.444155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:35.444155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:35.444155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:35.444155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:35.444155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:35.444155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:35.444155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:35.444155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:35.444155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:35.444155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:35.444155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:35.444155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:35.445026 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:45:35.445026 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:45:35.445026 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:45:35.445026 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:45:35.445026 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:45:35.445026 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:45:35.445026 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:35.445026 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:35.447477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 459/913] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:45:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:35.453903 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfeeaa0-dc3f-4126-92dd-4a57d438e23f None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:35.486840 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:35.541487 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eff682-47e3-40f3-b2b4-57252af10eac None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:35.550787 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:35.584668 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d0c8b-e5aa-48e4-93ac-a5f9239825d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:35.585696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d0c8b-e5aa-48e4-93ac-a5f9239825d4 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:35.586350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d0c8b-e5aa-48e4-93ac-a5f9239825d4 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:35.586951 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d0c8b-e5aa-48e4-93ac-a5f9239825d4 None admin] PATH_INFO: `/v3/services` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:35.587806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d0c8b-e5aa-48e4-93ac-a5f9239825d4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:35.588310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d0c8b-e5aa-48e4-93ac-a5f9239825d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:35.590508 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d0c8b-e5aa-48e4-93ac-a5f9239825d4 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:35.592924 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfeeaa0-dc3f-4126-92dd-4a57d438e23f None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:35.601424 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 455/914] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 31 10:45:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:35.603058 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d0c8b-e5aa-48e4-93ac-a5f9239825d4 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:35.644832 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f1df0-46b9-453f-82e5-ed28415eb910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:35.645688 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f1df0-46b9-453f-82e5-ed28415eb910 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:35.646276 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f1df0-46b9-453f-82e5-ed28415eb910 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:35.646587 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f1df0-46b9-453f-82e5-ed28415eb910 None admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:35.647407 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5f1df0-46b9-453f-82e5-ed28415eb910 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:35.647815 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5f1df0-46b9-453f-82e5-ed28415eb910 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:35.649612 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5f1df0-46b9-453f-82e5-ed28415eb910 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:35.670284 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 460/915] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:45:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:35.671645 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f1df0-46b9-453f-82e5-ed28415eb910 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:35.740943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d0c8b-e5aa-48e4-93ac-a5f9239825d4 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:35.779684 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:35.791448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7362f-fb0c-45fc-90a5-f083bd11b4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:35.791448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7362f-fb0c-45fc-90a5-f083bd11b4ea None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:35.791448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7362f-fb0c-45fc-90a5-f083bd11b4ea None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:35.791448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7362f-fb0c-45fc-90a5-f083bd11b4ea None admin] PATH_INFO: `/v3/services` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:35.803212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d7362f-fb0c-45fc-90a5-f083bd11b4ea None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:35.803212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d7362f-fb0c-45fc-90a5-f083bd11b4ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:35.803212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d7362f-fb0c-45fc-90a5-f083bd11b4ea None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:35.803438 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f1df0-46b9-453f-82e5-ed28415eb910 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:35.803528 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 456/916] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 31 10:45:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:35.803528 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9e025-4abf-4bb4-91a1-069ad8ab1ce6 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:35.803528 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9e025-4abf-4bb4-91a1-069ad8ab1ce6 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:35.803528 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9e025-4abf-4bb4-91a1-069ad8ab1ce6 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:35.803528 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 457/917] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:35.806530 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9e025-4abf-4bb4-91a1-069ad8ab1ce6 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:35.821759 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:35.841153 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd201d-6d79-4118-90ee-e3ef9d821524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:35.841810 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd201d-6d79-4118-90ee-e3ef9d821524 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:35.842094 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd201d-6d79-4118-90ee-e3ef9d821524 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:35.842366 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd201d-6d79-4118-90ee-e3ef9d821524 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:35.844750 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd201d-6d79-4118-90ee-e3ef9d821524 None admin] RBAC: Adding query filter params (role.id=1715f4cc50354636b1f7c3a9864e1e80, scope.project.id=8db9b3f8a9f041938ac7b39003a35015, user.id=7796ebff70054de090e02c58bf7000f4) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:35.844750 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd201d-6d79-4118-90ee-e3ef9d821524 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:35.844926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd201d-6d79-4118-90ee-e3ef9d821524 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:35.851667 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 461/918] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:45:35 2022] GET /identity/v3/role_assignments?user.id=7796ebff70054de090e02c58bf7000f4&scope.project.id=8db9b3f8a9f041938ac7b39003a35015&role.id=1715f4cc50354636b1f7c3a9864e1e80 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:35.852992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb78277-ac2e-434d-9368-9981d833c906 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:35.853271 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb78277-ac2e-434d-9368-9981d833c906 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:35.853540 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb78277-ac2e-434d-9368-9981d833c906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:35.884508 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-9eb78277-ac2e-434d-9368-9981d833c906 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:35.886257 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-9eb78277-ac2e-434d-9368-9981d833c906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:35.936253 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9e025-4abf-4bb4-91a1-069ad8ab1ce6 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:35.982123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c5b523-83ee-4bfa-9613-a9612313b59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:35.982971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5b523-83ee-4bfa-9613-a9612313b59e None admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:35.983353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5b523-83ee-4bfa-9613-a9612313b59e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:35.983723 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5b523-83ee-4bfa-9613-a9612313b59e None admin] PATH_INFO: `/v3/endpoints` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:35.983971 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 462/919] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:45:35.984648 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c5b523-83ee-4bfa-9613-a9612313b59e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:35.986234 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c5b523-83ee-4bfa-9613-a9612313b59e None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:36.116400 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 458/920] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 10:45:35 2022] POST /identity/v3/endpoints => generated 346 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:45:36.267630 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb78277-ac2e-434d-9368-9981d833c906 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:36.378505 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb78277-ac2e-434d-9368-9981d833c906 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:36.398322 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:36.427066 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10b372-52c1-40ec-b70d-7ae1c6ea346f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:36.428485 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10b372-52c1-40ec-b70d-7ae1c6ea346f None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:36.428960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10b372-52c1-40ec-b70d-7ae1c6ea346f None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:36.429453 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10b372-52c1-40ec-b70d-7ae1c6ea346f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:36.430253 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc10b372-52c1-40ec-b70d-7ae1c6ea346f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:36.431868 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc10b372-52c1-40ec-b70d-7ae1c6ea346f None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:36.443150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-fc10b372-52c1-40ec-b70d-7ae1c6ea346f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:45:36.443150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:36.443150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:45:36.443150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:45:36.443150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:45:36.443150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:45:36.443150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:45:36.443150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:36.443150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:45:36.443150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:36.443150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:45:36.443150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:36.443150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:36.443150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:36.443150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:36.443150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:36.443150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:36.444199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:36.444199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:36.444199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:36.444199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:36.444199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:36.444199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:36.444199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:45:36.444199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:45:36.444199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:45:36.444199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:45:36.444199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:36.444199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:36.444199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:36.444199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:36.444199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:36.444199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:36.445032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:36.445032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:36.445032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:36.445032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:36.445032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:36.445032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:36.445032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:36.445032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:36.445032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:36.445032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:36.445032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:45:36.445032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:36.445032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:45:36.445032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:36.447811 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 463/921] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Aug 31 10:45:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:36.456722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c5b523-83ee-4bfa-9613-a9612313b59e None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:36.530917 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:36.567275 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c5b523-83ee-4bfa-9613-a9612313b59e None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:36.604157 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb267ac-5d77-4dcf-aec0-2b786a91098c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:36.605698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb267ac-5d77-4dcf-aec0-2b786a91098c None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:36.606254 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb267ac-5d77-4dcf-aec0-2b786a91098c None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:36.606724 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb267ac-5d77-4dcf-aec0-2b786a91098c None admin] PATH_INFO: `/v3/services` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:36.607532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb267ac-5d77-4dcf-aec0-2b786a91098c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:36.607996 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb267ac-5d77-4dcf-aec0-2b786a91098c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:36.609519 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb267ac-5d77-4dcf-aec0-2b786a91098c None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:36.615252 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 459/922] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Aug 31 10:45:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:36.621784 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10b372-52c1-40ec-b70d-7ae1c6ea346f None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:36.732913 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10b372-52c1-40ec-b70d-7ae1c6ea346f None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:36.769627 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe890a-41a2-493a-b322-1e9ac02761bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:36.770572 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe890a-41a2-493a-b322-1e9ac02761bf None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:36.771003 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe890a-41a2-493a-b322-1e9ac02761bf None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:36.771427 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe890a-41a2-493a-b322-1e9ac02761bf None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:36.772236 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe890a-41a2-493a-b322-1e9ac02761bf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:36.772791 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe890a-41a2-493a-b322-1e9ac02761bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:36.774318 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe890a-41a2-493a-b322-1e9ac02761bf None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:36.780855 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 464/923] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Aug 31 10:45:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:36.786630 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb267ac-5d77-4dcf-aec0-2b786a91098c None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:36.897883 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb267ac-5d77-4dcf-aec0-2b786a91098c None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:36.913354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:36.935195 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc07989-9eaf-454d-b7d6-0dade73767f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:36.935809 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc07989-9eaf-454d-b7d6-0dade73767f3 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:36.936087 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc07989-9eaf-454d-b7d6-0dade73767f3 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:36.936362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc07989-9eaf-454d-b7d6-0dade73767f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:36.937076 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc07989-9eaf-454d-b7d6-0dade73767f3 None admin] RBAC: Adding query filter params (interface=public, service_id=001af680fba24503affd9ed78b36e68b, region_id=RegionOne) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:36.937384 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc07989-9eaf-454d-b7d6-0dade73767f3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:36.938780 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc07989-9eaf-454d-b7d6-0dade73767f3 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:36.947058 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 460/924] 10.222.0.16 () {56 vars in 1251 bytes} [Wed Aug 31 10:45:36 2022] GET /identity/v3/endpoints?service_id=001af680fba24503affd9ed78b36e68b&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:36.955512 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe890a-41a2-493a-b322-1e9ac02761bf None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:37.002339 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:37.068147 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe890a-41a2-493a-b322-1e9ac02761bf None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:37.107836 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab905a-5507-431a-813e-be9092ed4ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:37.108575 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab905a-5507-431a-813e-be9092ed4ef3 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:37.108810 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab905a-5507-431a-813e-be9092ed4ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:37.109075 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab905a-5507-431a-813e-be9092ed4ef3 None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:37.109696 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab905a-5507-431a-813e-be9092ed4ef3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:37.111111 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab905a-5507-431a-813e-be9092ed4ef3 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:37.117967 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 465/925] 10.222.0.16 () {56 vars in 1087 bytes} [Wed Aug 31 10:45:36 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:45:37.171851 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667fb23-be6f-467d-8a3a-18dd7f693971 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:37.172210 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667fb23-be6f-467d-8a3a-18dd7f693971 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:37.172409 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667fb23-be6f-467d-8a3a-18dd7f693971 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:37.174212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 461/926] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:37.180161 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca77bd-597b-4e38-b8a1-d2fbf78e30a5 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:37.180479 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca77bd-597b-4e38-b8a1-d2fbf78e30a5 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:37.180716 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca77bd-597b-4e38-b8a1-d2fbf78e30a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:37.201558 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-5bca77bd-597b-4e38-b8a1-d2fbf78e30a5 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:37.202852 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-5bca77bd-597b-4e38-b8a1-d2fbf78e30a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:37.283625 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 466/927] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:45:37.557042 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5667fb23-be6f-467d-8a3a-18dd7f693971 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:37.666325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5667fb23-be6f-467d-8a3a-18dd7f693971 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:37.702773 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d067f-4345-4f97-8b2c-4b5bd5ec830d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:37.703515 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d067f-4345-4f97-8b2c-4b5bd5ec830d None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:37.703796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d067f-4345-4f97-8b2c-4b5bd5ec830d None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:37.704070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d067f-4345-4f97-8b2c-4b5bd5ec830d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:37.704661 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d067f-4345-4f97-8b2c-4b5bd5ec830d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:37.706075 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d067f-4345-4f97-8b2c-4b5bd5ec830d None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:37.715353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-9d8d067f-4345-4f97-8b2c-4b5bd5ec830d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 10:45:37.715353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:37.715353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:45:37.715353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:45:37.715353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:45:37.715353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:45:37.715353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:45:37.715353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:37.715353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 10:45:37.715353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:37.715353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:45:37.715353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:37.715353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:37.715353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:37.715353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:37.715353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:37.716285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:37.716285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:37.716285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:37.716285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:37.716285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:37.716285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:37.716285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:37.716285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:45:37.716285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:45:37.716285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:45:37.716285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:45:37.716285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:37.716285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:37.716285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:37.716285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:37.716285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:37.716285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:37.718305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:37.718305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:37.718305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:37.718305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:37.718305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:37.718305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:37.718305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:37.718305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:37.718305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:37.718305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:37.718305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:45:37.718305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:37.718305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 10:45:37.718305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:37.718305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 462/928] 10.222.0.16 () {56 vars in 1132 bytes} [Wed Aug 31 10:45:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:37.722613 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca77bd-597b-4e38-b8a1-d2fbf78e30a5 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:37.759204 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:37.832459 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca77bd-597b-4e38-b8a1-d2fbf78e30a5 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:37.867866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab42858-b88e-4589-8587-34fbebb21316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:37.868479 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab42858-b88e-4589-8587-34fbebb21316 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:37.868755 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab42858-b88e-4589-8587-34fbebb21316 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:37.869280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab42858-b88e-4589-8587-34fbebb21316 None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:37.869909 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab42858-b88e-4589-8587-34fbebb21316 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:37.870269 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab42858-b88e-4589-8587-34fbebb21316 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:37.872285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab42858-b88e-4589-8587-34fbebb21316 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:37.877622 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 467/929] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Aug 31 10:45:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:37.882977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d067f-4345-4f97-8b2c-4b5bd5ec830d None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:37.916730 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ac91c-622e-4712-9520-734aad196ac9 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:37.917668 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ac91c-622e-4712-9520-734aad196ac9 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:37.917846 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ac91c-622e-4712-9520-734aad196ac9 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:37.919450 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 468/930] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:45:37.924320 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2537f-5193-4b85-8974-3064ee24b3bf None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:37.924598 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2537f-5193-4b85-8974-3064ee24b3bf None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:37.924867 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2537f-5193-4b85-8974-3064ee24b3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:37.931543 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:37.944137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-dfd2537f-5193-4b85-8974-3064ee24b3bf None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:37.945172 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-dfd2537f-5193-4b85-8974-3064ee24b3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:38.004863 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d067f-4345-4f97-8b2c-4b5bd5ec830d None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:38.021428 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 469/931] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:45:38.051043 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee929ebf-60fd-4bf2-a337-3cb450c3bb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:38.051658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee929ebf-60fd-4bf2-a337-3cb450c3bb7c None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:38.051936 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee929ebf-60fd-4bf2-a337-3cb450c3bb7c None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:38.052205 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee929ebf-60fd-4bf2-a337-3cb450c3bb7c None admin] PATH_INFO: `/v3/services` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:38.052887 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee929ebf-60fd-4bf2-a337-3cb450c3bb7c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:38.053307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee929ebf-60fd-4bf2-a337-3cb450c3bb7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:38.054817 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee929ebf-60fd-4bf2-a337-3cb450c3bb7c None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:38.061182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 463/932] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Aug 31 10:45:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:38.066909 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2537f-5193-4b85-8974-3064ee24b3bf None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:38.102320 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:38.176436 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2537f-5193-4b85-8974-3064ee24b3bf None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:38.210656 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9b1eb-1b9c-4369-a0f3-19fa6c1132e9 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:38.210963 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9b1eb-1b9c-4369-a0f3-19fa6c1132e9 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:38.211210 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9b1eb-1b9c-4369-a0f3-19fa6c1132e9 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:38.212627 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb955b8-1533-4d6a-8fbd-0fdfb7e5ac30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:38.213461 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb955b8-1533-4d6a-8fbd-0fdfb7e5ac30 None admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:38.213932 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 464/933] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:38.214214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb955b8-1533-4d6a-8fbd-0fdfb7e5ac30 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:38.214787 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb955b8-1533-4d6a-8fbd-0fdfb7e5ac30 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:38.215703 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb955b8-1533-4d6a-8fbd-0fdfb7e5ac30 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:38.217295 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb955b8-1533-4d6a-8fbd-0fdfb7e5ac30 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:38.218383 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94b91a-b9bb-4512-b063-63fe05aa84bd None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:38.218695 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94b91a-b9bb-4512-b063-63fe05aa84bd None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:38.218967 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94b91a-b9bb-4512-b063-63fe05aa84bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:38.239023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-4f94b91a-b9bb-4512-b063-63fe05aa84bd None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:38.240235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-4f94b91a-b9bb-4512-b063-63fe05aa84bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:38.317266 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 465/934] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:45:38.326418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 470/935] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 10:45:38 2022] POST /identity/v3/endpoints => generated 347 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:45:38.339092 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94b91a-b9bb-4512-b063-63fe05aa84bd None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:38.380036 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:38.461543 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94b91a-b9bb-4512-b063-63fe05aa84bd None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:38.507004 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040316d9-0fef-4be9-b486-cc0d6793e463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:38.507648 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040316d9-0fef-4be9-b486-cc0d6793e463 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:38.507923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040316d9-0fef-4be9-b486-cc0d6793e463 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:38.508192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040316d9-0fef-4be9-b486-cc0d6793e463 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:38.519054 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040316d9-0fef-4be9-b486-cc0d6793e463 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:45:38.524011 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-040316d9-0fef-4be9-b486-cc0d6793e463 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 10:45:38.524011 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:38.524011 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:38.524011 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:38.524011 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:38.524011 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:38.524011 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:38.524011 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:38.524011 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:38.524011 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:38.524011 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:38.524011 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:38.524011 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:45:38.524011 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:45:38.524011 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:45:38.524011 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:45:38.524935 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:45:38.524935 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:45:38.524935 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:45:38.524935 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:45:38.524935 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:38.524935 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:38.524935 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:38.524935 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:38.524935 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:38.524935 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:38.524935 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:38.524935 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:38.524935 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:38.524935 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:38.524935 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:38.524935 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:38.525762 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:38.525762 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:38.525762 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:38.525762 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:38.525762 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:45:38.525762 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:45:38.525762 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:45:38.525762 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:45:38.525762 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:45:38.525762 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:45:38.525762 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 10:45:38.525762 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:38.525762 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 466/936] 10.222.0.16 () {56 vars in 1120 bytes} [Wed Aug 31 10:45:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:38.532704 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb955b8-1533-4d6a-8fbd-0fdfb7e5ac30 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:38.593878 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040316d9-0fef-4be9-b486-cc0d6793e463 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:38.631632 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:38.649645 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb955b8-1533-4d6a-8fbd-0fdfb7e5ac30 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:38.691077 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449e73a-f25c-4123-b20a-a752c125d2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:38.691720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449e73a-f25c-4123-b20a-a752c125d2f0 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:38.692006 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449e73a-f25c-4123-b20a-a752c125d2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:38.692339 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449e73a-f25c-4123-b20a-a752c125d2f0 None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:38.693026 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f449e73a-f25c-4123-b20a-a752c125d2f0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:38.693388 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f449e73a-f25c-4123-b20a-a752c125d2f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:38.694859 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f449e73a-f25c-4123-b20a-a752c125d2f0 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:38.703748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:38.704534 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 471/937] 10.222.0.16 () {56 vars in 1115 bytes} [Wed Aug 31 10:45:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:38.710341 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449e73a-f25c-4123-b20a-a752c125d2f0 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:38.719708 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040316d9-0fef-4be9-b486-cc0d6793e463 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:38.772384 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a906e48-de26-4d24-b0df-5e60a5c29674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:38.773285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a906e48-de26-4d24-b0df-5e60a5c29674 None admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:38.773625 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a906e48-de26-4d24-b0df-5e60a5c29674 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:38.773964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a906e48-de26-4d24-b0df-5e60a5c29674 None admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:38.775031 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a906e48-de26-4d24-b0df-5e60a5c29674 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:38.776580 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a906e48-de26-4d24-b0df-5e60a5c29674 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:38.779786 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: WARNING keystone.server.flask.common [None req-6a906e48-de26-4d24-b0df-5e60a5c29674 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 31 10:45:38.830473 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.sql.core [None req-6a906e48-de26-4d24-b0df-5e60a5c29674 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 10:45:38.830473 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 10:45:38.830473 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [parameters: {'id': '762cb0e0c7b54cf79718e5358d7635b2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 10:45:38.830473 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=82089) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 10:45:38.838358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-6a906e48-de26-4d24-b0df-5e60a5c29674 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 762cb0e0c7b54cf79718e5358d7635b2.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 762cb0e0c7b54cf79718e5358d7635b2. Aug 31 10:45:38.838358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:38.838358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 10:45:38.838358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 10:45:38.838358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 10:45:38.838358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 10:45:38.838358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 10:45:38.838358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 10:45:38.838358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 10:45:38.838358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application conn.query(q) Aug 31 10:45:38.838358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 10:45:38.838358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 10:45:38.838358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 10:45:38.839871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application result.read() Aug 31 10:45:38.839871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 10:45:38.839871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 10:45:38.839871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 10:45:38.839871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 10:45:38.839871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 10:45:38.839871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 10:45:38.839871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 10:45:38.839871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 10:45:38.839871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 10:45:38.839871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:38.839871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 10:45:38.839871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:38.839871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:38.839871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 10:45:38.839871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 10:45:38.839871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 31 10:45:38.839871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 31 10:45:38.841172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 10:45:38.841172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application next(self.gen) Aug 31 10:45:38.841172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 10:45:38.841172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application yield resource Aug 31 10:45:38.841172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 10:45:38.841172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application next(self.gen) Aug 31 10:45:38.841172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 10:45:38.841172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 10:45:38.841172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 10:45:38.841172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 10:45:38.841172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 10:45:38.841172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise self.value Aug 31 10:45:38.841172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 10:45:38.841172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 10:45:38.841172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 10:45:38.841172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application session.commit() Aug 31 10:45:38.841172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 10:45:38.842044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 10:45:38.842044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 10:45:38.842044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 10:45:38.842044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 10:45:38.842044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self.session.flush() Aug 31 10:45:38.842044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 10:45:38.842044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 10:45:38.842044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 10:45:38.842044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 10:45:38.842044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 10:45:38.842044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application compat.raise_( Aug 31 10:45:38.842044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 10:45:38.842044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception Aug 31 10:45:38.842044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 10:45:38.842044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 10:45:38.842044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 10:45:38.842044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 10:45:38.844977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 10:45:38.844977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 10:45:38.844977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 10:45:38.844977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 10:45:38.844977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 10:45:38.844977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 10:45:38.844977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 10:45:38.844977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 10:45:38.844977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 10:45:38.844977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 10:45:38.844977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 10:45:38.844977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 10:45:38.844977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 10:45:38.844977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 10:45:38.844977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 10:45:38.845732 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 10:45:38.845732 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 10:45:38.845732 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception Aug 31 10:45:38.845732 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 10:45:38.845732 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 10:45:38.845732 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 10:45:38.845732 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 10:45:38.845732 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 10:45:38.845732 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 10:45:38.845732 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 10:45:38.845732 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application conn.query(q) Aug 31 10:45:38.845732 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 10:45:38.845732 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 10:45:38.845732 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 10:45:38.845732 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application result.read() Aug 31 10:45:38.845732 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 10:45:38.845732 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 10:45:38.846609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 10:45:38.846609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 10:45:38.846609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 10:45:38.846609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 10:45:38.846609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 10:45:38.846609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 10:45:38.846609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 10:45:38.846609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 10:45:38.846609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [parameters: {'id': '762cb0e0c7b54cf79718e5358d7635b2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 10:45:38.846609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 10:45:38.846609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:38.846609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:45:38.846609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:38.846609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:38.846609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 31 10:45:38.847382 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 31 10:45:38.847382 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 10:45:38.847382 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 10:45:38.847382 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 31 10:45:38.847382 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:38.847382 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:45:38.847382 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:38.847382 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:38.847382 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:38.847382 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:38.847382 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:38.847382 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:38.847382 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:38.847382 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:38.847382 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:38.847382 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:38.847382 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:38.849181 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:38.849181 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 31 10:45:38.849181 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 31 10:45:38.849181 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:38.849181 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:38.849181 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 31 10:45:38.849181 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 31 10:45:38.849181 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 762cb0e0c7b54cf79718e5358d7635b2. Aug 31 10:45:38.849181 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:38.849181 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 467/938] 10.222.0.16 () {60 vars in 1140 bytes} [Wed Aug 31 10:45:38 2022] POST /identity/v3/projects => generated 262 bytes in 247 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:38.849181 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a906e48-de26-4d24-b0df-5e60a5c29674 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:38.849758 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449e73a-f25c-4123-b20a-a752c125d2f0 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:38.886586 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3334e17-99d8-4f9a-b41f-980627c76866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:38.887399 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3334e17-99d8-4f9a-b41f-980627c76866 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:38.887681 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3334e17-99d8-4f9a-b41f-980627c76866 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:38.887956 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3334e17-99d8-4f9a-b41f-980627c76866 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:38.899845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3334e17-99d8-4f9a-b41f-980627c76866 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:38.901349 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3334e17-99d8-4f9a-b41f-980627c76866 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:38.911050 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:38.919433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-e3334e17-99d8-4f9a-b41f-980627c76866 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:38.919433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:38.919433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:38.919433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:38.919433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:38.919433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:38.919433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:38.919433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:38.919433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:38.919433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:38.919433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:38.919433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:38.919433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:38.919433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:38.919433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:38.919433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:38.925486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:38.925486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:38.925486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:38.925486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:38.925486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:38.925486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:38.925486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:38.925486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:38.925486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:38.925486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:38.925486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:38.925486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:38.925486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:38.925486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:38.925486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:38.925486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:38.925486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:38.926941 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:38.926941 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:38.926941 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:38.926941 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 472/939] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:38.928834 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3334e17-99d8-4f9a-b41f-980627c76866 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:38.962962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a906e48-de26-4d24-b0df-5e60a5c29674 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:39.002644 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4559c001-baa9-4ab3-8922-ea712831387e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:39.003467 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559c001-baa9-4ab3-8922-ea712831387e None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:39.003668 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559c001-baa9-4ab3-8922-ea712831387e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:39.004050 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559c001-baa9-4ab3-8922-ea712831387e None admin] PATH_INFO: `/v3/projects/service` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:39.011167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4559c001-baa9-4ab3-8922-ea712831387e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:39.012975 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4559c001-baa9-4ab3-8922-ea712831387e None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:39.024806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-4559c001-baa9-4ab3-8922-ea712831387e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:39.024806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:39.024806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:39.024806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:39.024806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:39.024806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:39.024806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:39.024806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:39.024806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:39.024806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:39.024806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:39.024806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:39.024806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:45:39.024806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:45:39.024806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:45:39.025750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:45:39.025750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:39.025750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:39.025750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:39.025750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:39.025750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:39.025750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:39.025750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:39.025750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:39.025750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:39.025750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:39.025750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:39.025750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:39.025750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:39.025750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:39.025750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:39.025750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:39.027385 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:45:39.027385 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:45:39.027385 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:45:39.027385 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:45:39.027385 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:45:39.027385 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:45:39.027385 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:39.027385 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:39.027385 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 468/940] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:45:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:39.032442 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4559c001-baa9-4ab3-8922-ea712831387e None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:39.043261 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3334e17-99d8-4f9a-b41f-980627c76866 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:39.044197 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:39.078719 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8f4e1-20b1-4867-82be-b51b78e6e2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:39.079715 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8f4e1-20b1-4867-82be-b51b78e6e2ae None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:39.080199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8f4e1-20b1-4867-82be-b51b78e6e2ae None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:39.080716 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8f4e1-20b1-4867-82be-b51b78e6e2ae None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:39.084606 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd8f4e1-20b1-4867-82be-b51b78e6e2ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:39.085346 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd8f4e1-20b1-4867-82be-b51b78e6e2ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:39.086961 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd8f4e1-20b1-4867-82be-b51b78e6e2ae None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:39.117169 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 473/941] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:39.123424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8f4e1-20b1-4867-82be-b51b78e6e2ae None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:39.153729 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4559c001-baa9-4ab3-8922-ea712831387e None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:39.191317 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:39.197598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cf259-74c3-4019-bf98-b91c7e8cc159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:39.198781 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cf259-74c3-4019-bf98-b91c7e8cc159 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:39.204939 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cf259-74c3-4019-bf98-b91c7e8cc159 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:39.204939 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cf259-74c3-4019-bf98-b91c7e8cc159 None admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:39.204939 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1cf259-74c3-4019-bf98-b91c7e8cc159 None admin] RBAC: Adding query filter params (name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:39.204939 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1cf259-74c3-4019-bf98-b91c7e8cc159 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:39.204939 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1cf259-74c3-4019-bf98-b91c7e8cc159 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:39.222622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 469/942] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Aug 31 10:45:39 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:39.224577 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e62029-9291-4910-a4ca-e13ed27498e3 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:39.224861 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e62029-9291-4910-a4ca-e13ed27498e3 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:39.225129 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e62029-9291-4910-a4ca-e13ed27498e3 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:39.226645 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 470/943] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:39.232290 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc14bc-d1d5-481b-85f2-da4390b7db7e None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:39.232596 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc14bc-d1d5-481b-85f2-da4390b7db7e None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:39.232841 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc14bc-d1d5-481b-85f2-da4390b7db7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:39.247794 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8f4e1-20b1-4867-82be-b51b78e6e2ae None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:39.255285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-74fc14bc-d1d5-481b-85f2-da4390b7db7e None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:39.257036 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-74fc14bc-d1d5-481b-85f2-da4390b7db7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:39.299827 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f0ba7-1245-4f76-a28c-a09802ee1c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:39.301138 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f0ba7-1245-4f76-a28c-a09802ee1c8d None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:39.301670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f0ba7-1245-4f76-a28c-a09802ee1c8d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:39.302861 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f0ba7-1245-4f76-a28c-a09802ee1c8d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:39.312269 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f0ba7-1245-4f76-a28c-a09802ee1c8d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:39.313790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f0ba7-1245-4f76-a28c-a09802ee1c8d None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:39.327347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-0d6f0ba7-1245-4f76-a28c-a09802ee1c8d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 10:45:39.327347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:39.327347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:39.327347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:39.327347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:39.327347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:39.327347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:39.327347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:39.327347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:39.327347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:39.327347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:39.327347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:39.327347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:45:39.327347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:45:39.327347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:45:39.327347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:45:39.328356 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:39.328356 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:39.328356 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:45:39.328356 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:39.328356 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:45:39.328356 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:39.328356 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:39.328356 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:39.328356 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:39.328356 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:39.328356 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:39.328356 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:39.328356 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:39.328356 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:39.328356 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:39.328356 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:39.328356 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:39.328356 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:39.329268 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:39.329268 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:39.329268 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:45:39.329268 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:45:39.329268 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:45:39.329268 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:45:39.329268 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:45:39.329268 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:45:39.329268 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 10:45:39.329268 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:39.333963 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 474/944] 10.222.0.16 () {56 vars in 1117 bytes} [Wed Aug 31 10:45:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:39.334974 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f0ba7-1245-4f76-a28c-a09802ee1c8d None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:39.342836 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:39.350092 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 471/945] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:45:39.447553 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f0ba7-1245-4f76-a28c-a09802ee1c8d None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:39.469520 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:39.484564 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a5ab6-c7b2-42a6-b1d7-3484ac02c131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:39.485260 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a5ab6-c7b2-42a6-b1d7-3484ac02c131 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:39.485655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a5ab6-c7b2-42a6-b1d7-3484ac02c131 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:39.485999 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a5ab6-c7b2-42a6-b1d7-3484ac02c131 None admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:39.486837 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a5ab6-c7b2-42a6-b1d7-3484ac02c131 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:39.487203 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a5ab6-c7b2-42a6-b1d7-3484ac02c131 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:39.488695 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a5ab6-c7b2-42a6-b1d7-3484ac02c131 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:39.507790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 475/946] 10.222.0.16 () {56 vars in 1149 bytes} [Wed Aug 31 10:45:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:39.514026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc14bc-d1d5-481b-85f2-da4390b7db7e None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:39.620039 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc14bc-d1d5-481b-85f2-da4390b7db7e None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:39.631635 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a5ab6-c7b2-42a6-b1d7-3484ac02c131 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:39.658848 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae5cf0-f717-4845-966f-2318cbad5b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:39.659579 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae5cf0-f717-4845-966f-2318cbad5b7b None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:39.659922 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae5cf0-f717-4845-966f-2318cbad5b7b None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:39.660259 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae5cf0-f717-4845-966f-2318cbad5b7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:39.670905 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae5cf0-f717-4845-966f-2318cbad5b7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:39.672431 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae5cf0-f717-4845-966f-2318cbad5b7b None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:39.689496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-0dae5cf0-f717-4845-966f-2318cbad5b7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:39.689496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:39.689496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:39.689496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:39.689496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:39.689496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:39.689496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:39.689496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:39.689496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:39.689496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:39.689496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:39.689496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:39.689496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:39.689496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:39.689496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:39.689496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:39.690650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:39.690650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:39.690650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:39.690650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:39.690650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:39.690650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:39.690650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:39.690650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:39.690650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:39.690650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:39.690650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:39.690650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:39.690650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:39.690650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:39.690650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:39.690650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:39.690650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:39.691542 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:39.691542 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:39.691542 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:39.695983 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 472/947] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:39.697166 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae5cf0-f717-4845-966f-2318cbad5b7b None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:39.750914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:39.778188 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a5ab6-c7b2-42a6-b1d7-3484ac02c131 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:39.814414 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae5cf0-f717-4845-966f-2318cbad5b7b None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:39.814639 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb3c71-be76-416b-812d-4103aa547ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:39.815175 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb3c71-be76-416b-812d-4103aa547ede None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:39.815453 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb3c71-be76-416b-812d-4103aa547ede None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:39.815722 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb3c71-be76-416b-812d-4103aa547ede None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:39.816305 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb3c71-be76-416b-812d-4103aa547ede None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:39.817653 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb3c71-be76-416b-812d-4103aa547ede None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:39.827177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-bfdb3c71-be76-416b-812d-4103aa547ede None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:45:39.827177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:39.827177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:45:39.827177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:45:39.827177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:45:39.827177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:45:39.827177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:45:39.827177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:39.827177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:45:39.827177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:39.827177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:45:39.827177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:39.827177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:39.827177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:39.827177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:39.827177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:39.827177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:39.828224 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:39.828224 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:39.828224 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:39.828224 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:39.828224 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:39.828224 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:39.828224 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:45:39.828224 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:45:39.828224 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:45:39.828224 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:45:39.828224 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:39.828224 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:39.828224 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:39.828224 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:39.828224 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:39.828224 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:39.829091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:39.829091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:39.829091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:39.829091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:39.829091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:39.829091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:39.829091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:39.829091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:39.829091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:39.829091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:39.829091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:45:39.829091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:39.829091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:45:39.829091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:39.829091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 476/948] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Aug 31 10:45:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:39.835595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb3c71-be76-416b-812d-4103aa547ede None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:39.862476 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256bc585-b93a-490d-b63e-b783c2661ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:39.863232 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bc585-b93a-490d-b63e-b783c2661ea0 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:39.863623 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bc585-b93a-490d-b63e-b783c2661ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:39.864031 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bc585-b93a-490d-b63e-b783c2661ea0 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:39.864773 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256bc585-b93a-490d-b63e-b783c2661ea0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:39.865276 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256bc585-b93a-490d-b63e-b783c2661ea0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:39.866766 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256bc585-b93a-490d-b63e-b783c2661ea0 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:39.885350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 473/949] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:39.887416 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256bc585-b93a-490d-b63e-b783c2661ea0 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:39.898894 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:39.954945 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb3c71-be76-416b-812d-4103aa547ede None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:39.992385 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256bc585-b93a-490d-b63e-b783c2661ea0 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:39.992980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ffea64-46a1-479a-b681-6b05997bc727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:39.993751 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ffea64-46a1-479a-b681-6b05997bc727 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:39.994553 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ffea64-46a1-479a-b681-6b05997bc727 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:39.994996 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ffea64-46a1-479a-b681-6b05997bc727 None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:39.996567 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ffea64-46a1-479a-b681-6b05997bc727 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:39.997010 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ffea64-46a1-479a-b681-6b05997bc727 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:39.998537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ffea64-46a1-479a-b681-6b05997bc727 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:40.008633 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:40.009064 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 477/950] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Aug 31 10:45:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:40.010887 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ffea64-46a1-479a-b681-6b05997bc727 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:40.031054 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc7bfb-3036-4e39-90a7-bc4710579529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:40.031901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc7bfb-3036-4e39-90a7-bc4710579529 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:40.032270 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc7bfb-3036-4e39-90a7-bc4710579529 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:40.032636 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc7bfb-3036-4e39-90a7-bc4710579529 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:40.040038 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc7bfb-3036-4e39-90a7-bc4710579529 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:40.041600 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc7bfb-3036-4e39-90a7-bc4710579529 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:40.054767 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-4ecc7bfb-3036-4e39-90a7-bc4710579529 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:40.054767 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:40.054767 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:40.054767 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:40.054767 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:40.054767 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:40.054767 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:40.054767 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:40.054767 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:40.054767 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:40.054767 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:40.054767 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:40.054767 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:45:40.054767 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:45:40.054767 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:45:40.055797 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:45:40.055797 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:40.055797 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:40.055797 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:40.055797 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:40.055797 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:40.055797 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:40.055797 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:40.055797 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:40.055797 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:40.055797 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:40.055797 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:40.055797 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:40.055797 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:40.055797 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:40.055797 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:40.055797 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:40.056745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:45:40.056745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:45:40.056745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:45:40.056745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:45:40.056745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:45:40.056745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:45:40.056745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:40.056745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:40.061589 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 474/951] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:45:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:40.062859 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc7bfb-3036-4e39-90a7-bc4710579529 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:40.145288 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ffea64-46a1-479a-b681-6b05997bc727 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:40.156558 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:40.174845 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc7bfb-3036-4e39-90a7-bc4710579529 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:40.202552 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149cc54-635f-4aeb-a680-cb62976f0079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:40.204002 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149cc54-635f-4aeb-a680-cb62976f0079 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:40.204486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149cc54-635f-4aeb-a680-cb62976f0079 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:40.205340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149cc54-635f-4aeb-a680-cb62976f0079 None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:40.206494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149cc54-635f-4aeb-a680-cb62976f0079 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:40.207391 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149cc54-635f-4aeb-a680-cb62976f0079 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:40.209524 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149cc54-635f-4aeb-a680-cb62976f0079 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:40.216087 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe384fa6-cbdb-4a75-ba3c-cbbb0930118c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:40.216702 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe384fa6-cbdb-4a75-ba3c-cbbb0930118c None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:40.216974 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe384fa6-cbdb-4a75-ba3c-cbbb0930118c None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:40.217242 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe384fa6-cbdb-4a75-ba3c-cbbb0930118c None admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:40.222436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe384fa6-cbdb-4a75-ba3c-cbbb0930118c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:40.222436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe384fa6-cbdb-4a75-ba3c-cbbb0930118c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:40.222436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe384fa6-cbdb-4a75-ba3c-cbbb0930118c None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:40.223161 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 478/952] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Aug 31 10:45:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:40.225433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149cc54-635f-4aeb-a680-cb62976f0079 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:40.241869 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 475/953] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:45:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:40.243600 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe384fa6-cbdb-4a75-ba3c-cbbb0930118c None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:40.279449 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:40.369882 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149cc54-635f-4aeb-a680-cb62976f0079 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:40.372212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe384fa6-cbdb-4a75-ba3c-cbbb0930118c None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:40.413819 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e0abf-1882-4279-b51c-d844784e4b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:40.415016 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e0abf-1882-4279-b51c-d844784e4b70 None admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:40.415488 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e0abf-1882-4279-b51c-d844784e4b70 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:40.415968 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e0abf-1882-4279-b51c-d844784e4b70 None admin] PATH_INFO: `/v3/projects/8db9b3f8a9f041938ac7b39003a35015/users/7796ebff70054de090e02c58bf7000f4/roles/1715f4cc50354636b1f7c3a9864e1e80` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:40.416370 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85674daa-ed9e-405e-859d-2ee007d4b27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:40.417807 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85674daa-ed9e-405e-859d-2ee007d4b27e None admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:40.418192 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85674daa-ed9e-405e-859d-2ee007d4b27e None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:40.418769 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85674daa-ed9e-405e-859d-2ee007d4b27e None admin] PATH_INFO: `/v3/endpoints` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:40.419129 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e0abf-1882-4279-b51c-d844784e4b70 None admin] RBAC: Authorizing `identity:create_grant(project_id=8db9b3f8a9f041938ac7b39003a35015, user_id=7796ebff70054de090e02c58bf7000f4, role_id=1715f4cc50354636b1f7c3a9864e1e80)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:40.419539 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85674daa-ed9e-405e-859d-2ee007d4b27e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:40.420570 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e0abf-1882-4279-b51c-d844784e4b70 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:40.420905 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85674daa-ed9e-405e-859d-2ee007d4b27e None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:40.514939 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 476/954] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 31 10:45:40 2022] PUT /identity/v3/projects/8db9b3f8a9f041938ac7b39003a35015/users/7796ebff70054de090e02c58bf7000f4/roles/1715f4cc50354636b1f7c3a9864e1e80 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:45:40.523119 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef8ec6-8b84-4096-a713-ec8d502e4252 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:40.523119 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef8ec6-8b84-4096-a713-ec8d502e4252 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:40.523403 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef8ec6-8b84-4096-a713-ec8d502e4252 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:40.524969 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 477/955] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:40.529933 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 479/956] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 10:45:40 2022] POST /identity/v3/endpoints => generated 349 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:45:40.531770 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f25102-ea18-4b38-98f4-150be1ee16ee None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:40.532276 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f25102-ea18-4b38-98f4-150be1ee16ee None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:40.532740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f25102-ea18-4b38-98f4-150be1ee16ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:40.555627 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-90f25102-ea18-4b38-98f4-150be1ee16ee None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:40.557392 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-90f25102-ea18-4b38-98f4-150be1ee16ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:40.686579 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 478/957] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:45:40.982126 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85674daa-ed9e-405e-859d-2ee007d4b27e None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:41.035286 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:41.134098 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85674daa-ed9e-405e-859d-2ee007d4b27e None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:41.171689 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e1d8b-42cb-4bc7-9e51-c8f1cc463aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:41.172680 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e1d8b-42cb-4bc7-9e51-c8f1cc463aee None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:41.173132 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e1d8b-42cb-4bc7-9e51-c8f1cc463aee None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:41.173592 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e1d8b-42cb-4bc7-9e51-c8f1cc463aee None admin] PATH_INFO: `/v3/services/compute` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:41.174682 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e1d8b-42cb-4bc7-9e51-c8f1cc463aee None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:41.177180 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e1d8b-42cb-4bc7-9e51-c8f1cc463aee None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:41.194818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-b74e1d8b-42cb-4bc7-9e51-c8f1cc463aee None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 10:45:41.194818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:41.194818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:45:41.194818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:45:41.194818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:45:41.194818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:45:41.194818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:45:41.194818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:41.194818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 10:45:41.194818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:41.194818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:45:41.194818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:41.194818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:41.194818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:41.194818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:41.194818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:41.194818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:41.195966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:41.195966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:41.195966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:41.195966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:41.195966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:41.195966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:41.195966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:45:41.195966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:45:41.195966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:45:41.195966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:45:41.195966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:41.195966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:41.195966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:41.195966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:41.195966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:41.195966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:41.196805 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:41.196805 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:41.196805 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:41.196805 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:41.196805 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:41.196805 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:41.196805 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:41.196805 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:41.196805 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:41.196805 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:41.196805 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:45:41.196805 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:41.196805 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 10:45:41.196805 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:41.197564 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 480/958] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:45:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:41.206959 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f25102-ea18-4b38-98f4-150be1ee16ee None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:41.235949 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:41.317843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f25102-ea18-4b38-98f4-150be1ee16ee None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:41.352217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddbd4c2-a8bb-4d86-8517-1453a1fea5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:41.352920 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbd4c2-a8bb-4d86-8517-1453a1fea5a8 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:41.353196 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbd4c2-a8bb-4d86-8517-1453a1fea5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:41.353468 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbd4c2-a8bb-4d86-8517-1453a1fea5a8 None admin] PATH_INFO: `/v3/services` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:41.354092 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddbd4c2-a8bb-4d86-8517-1453a1fea5a8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:41.354408 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddbd4c2-a8bb-4d86-8517-1453a1fea5a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:41.356398 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddbd4c2-a8bb-4d86-8517-1453a1fea5a8 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:41.361429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 479/959] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Aug 31 10:45:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:41.367099 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e1d8b-42cb-4bc7-9e51-c8f1cc463aee None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:41.462415 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:41.472534 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e1d8b-42cb-4bc7-9e51-c8f1cc463aee None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:41.508951 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b097cb-120d-4992-a4f5-79ef0ad0abf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:41.509477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b097cb-120d-4992-a4f5-79ef0ad0abf2 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:41.509752 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b097cb-120d-4992-a4f5-79ef0ad0abf2 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:41.510127 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b097cb-120d-4992-a4f5-79ef0ad0abf2 None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:41.510804 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b097cb-120d-4992-a4f5-79ef0ad0abf2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:41.511154 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b097cb-120d-4992-a4f5-79ef0ad0abf2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:41.512513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b097cb-120d-4992-a4f5-79ef0ad0abf2 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:41.517332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 481/960] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Aug 31 10:45:41 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:41.569468 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc9938-577e-488d-a5de-185538d4d395 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:41.569832 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc9938-577e-488d-a5de-185538d4d395 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:41.570000 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc9938-577e-488d-a5de-185538d4d395 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:41.571806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 480/961] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:41.577542 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42c414-a9a2-4727-9efc-addbc2301d52 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:41.577860 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42c414-a9a2-4727-9efc-addbc2301d52 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:41.578087 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42c414-a9a2-4727-9efc-addbc2301d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:41.597616 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-cb42c414-a9a2-4727-9efc-addbc2301d52 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:41.598890 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-cb42c414-a9a2-4727-9efc-addbc2301d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:41.678061 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 482/962] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:45:41.946716 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc9938-577e-488d-a5de-185538d4d395 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:42.033690 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:42.053979 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc9938-577e-488d-a5de-185538d4d395 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:42.089329 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccedd35-76f9-4fae-9209-537a58fbcdd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:42.090158 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccedd35-76f9-4fae-9209-537a58fbcdd8 None admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:42.090493 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccedd35-76f9-4fae-9209-537a58fbcdd8 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:42.090762 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccedd35-76f9-4fae-9209-537a58fbcdd8 None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:42.091410 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccedd35-76f9-4fae-9209-537a58fbcdd8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:42.098315 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccedd35-76f9-4fae-9209-537a58fbcdd8 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:42.148321 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.sql.core [None req-5ccedd35-76f9-4fae-9209-537a58fbcdd8 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 10:45:42.148321 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 10:45:42.148321 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [parameters: {'user_id': 'b204a7f7eb8a455c97269c522502cc1d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 10:45:42.148321 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=82089) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 10:45:42.155679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-5ccedd35-76f9-4fae-9209-537a58fbcdd8 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 10:45:42.155679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:42.155679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 10:45:42.155679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 10:45:42.155679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 10:45:42.155679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 10:45:42.155679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 10:45:42.155679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 10:45:42.155679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 10:45:42.155679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application conn.query(q) Aug 31 10:45:42.155679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 10:45:42.155679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 10:45:42.155679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 10:45:42.155679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application result.read() Aug 31 10:45:42.155679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 10:45:42.158793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 10:45:42.158793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 10:45:42.158793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 10:45:42.158793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 10:45:42.158793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 10:45:42.158793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 10:45:42.158793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 10:45:42.158793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 10:45:42.158793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:42.158793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 10:45:42.158793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:42.158793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:42.158793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 10:45:42.158793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 10:45:42.158793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 31 10:45:42.158793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 31 10:45:42.158793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 10:45:42.158793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application next(self.gen) Aug 31 10:45:42.159735 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 10:45:42.159735 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application yield resource Aug 31 10:45:42.159735 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 10:45:42.159735 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application next(self.gen) Aug 31 10:45:42.159735 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 10:45:42.159735 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 10:45:42.159735 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 10:45:42.159735 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 10:45:42.159735 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 10:45:42.159735 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise self.value Aug 31 10:45:42.159735 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 10:45:42.159735 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 10:45:42.159735 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 10:45:42.159735 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application session.commit() Aug 31 10:45:42.159735 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 10:45:42.159735 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 10:45:42.159735 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 10:45:42.160605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 10:45:42.160605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 10:45:42.160605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self.session.flush() Aug 31 10:45:42.160605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 10:45:42.160605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 10:45:42.160605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 10:45:42.160605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 10:45:42.160605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 10:45:42.160605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application compat.raise_( Aug 31 10:45:42.160605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 10:45:42.160605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception Aug 31 10:45:42.160605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 10:45:42.160605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 10:45:42.160605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 10:45:42.160605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 10:45:42.160605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 10:45:42.160605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 10:45:42.161576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 10:45:42.161576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 10:45:42.161576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 31 10:45:42.161576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 31 10:45:42.161576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 10:45:42.161576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 10:45:42.161576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 10:45:42.161576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 10:45:42.161576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 10:45:42.161576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 10:45:42.161576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 10:45:42.161576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 10:45:42.161576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 10:45:42.161576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 10:45:42.161576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 10:45:42.161576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception Aug 31 10:45:42.162441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 10:45:42.162441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 10:45:42.162441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 10:45:42.162441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 10:45:42.162441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 10:45:42.162441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 10:45:42.162441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 10:45:42.162441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application conn.query(q) Aug 31 10:45:42.162441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 10:45:42.162441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 10:45:42.162441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 10:45:42.162441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application result.read() Aug 31 10:45:42.162441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 10:45:42.162441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 10:45:42.162441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 10:45:42.162441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 10:45:42.164037 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 10:45:42.164037 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 10:45:42.164037 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 10:45:42.164037 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 10:45:42.164037 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 10:45:42.164037 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 10:45:42.164037 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [parameters: {'user_id': 'b204a7f7eb8a455c97269c522502cc1d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 10:45:42.164037 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 10:45:42.164037 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:42.164037 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:45:42.164037 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:42.164037 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:42.164037 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:42.164037 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:42.164037 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:42.165785 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:42.165785 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:42.165785 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:42.165785 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:42.165785 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:42.165785 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:42.165785 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:42.165785 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 31 10:45:42.165785 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 31 10:45:42.165785 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:42.165785 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:42.165785 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:45:42.165785 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:42.165785 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:45:42.165785 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:42.165785 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 31 10:45:42.165785 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 31 10:45:42.168770 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 31 10:45:42.168770 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 31 10:45:42.168770 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 10:45:42.168770 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 10:45:42.168770 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 10:45:42.168770 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:42.168770 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 481/963] 10.222.0.16 () {60 vars in 1131 bytes} [Wed Aug 31 10:45:41 2022] POST /identity/v3/users => generated 160 bytes in 211 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:42.169664 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb42c414-a9a2-4727-9efc-addbc2301d52 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:42.254876 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:42.270369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb42c414-a9a2-4727-9efc-addbc2301d52 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:42.317843 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ed3c5-546b-402e-8fc1-786c9bbf5c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:42.318542 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ed3c5-546b-402e-8fc1-786c9bbf5c66 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:42.318818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ed3c5-546b-402e-8fc1-786c9bbf5c66 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:42.319089 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ed3c5-546b-402e-8fc1-786c9bbf5c66 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:42.327748 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143ed3c5-546b-402e-8fc1-786c9bbf5c66 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:42.329187 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143ed3c5-546b-402e-8fc1-786c9bbf5c66 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:42.342262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-143ed3c5-546b-402e-8fc1-786c9bbf5c66 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 10:45:42.342262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:42.342262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:42.342262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:42.342262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:42.342262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:42.342262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:42.342262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:42.342262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:42.342262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:42.342262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:42.342262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:42.342262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:45:42.342262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:45:42.342262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:45:42.342262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:45:42.344503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:42.344503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:42.344503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:45:42.344503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:42.344503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:45:42.344503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:42.344503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:42.344503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:42.344503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:42.344503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:42.344503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:42.344503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:42.344503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:42.344503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:42.344503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:42.344503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:42.344503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:42.344503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:42.346734 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:42.346734 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:42.346734 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:45:42.346734 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:45:42.346734 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:45:42.346734 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:45:42.346734 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:45:42.346734 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:45:42.346734 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 10:45:42.346734 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:42.346734 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 483/964] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:45:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:42.354457 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccedd35-76f9-4fae-9209-537a58fbcdd8 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:42.458486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccedd35-76f9-4fae-9209-537a58fbcdd8 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:42.494825 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a62ec-572d-41e4-88bd-312a9efadfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:42.495626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a62ec-572d-41e4-88bd-312a9efadfe3 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:42.495890 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a62ec-572d-41e4-88bd-312a9efadfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:42.496143 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a62ec-572d-41e4-88bd-312a9efadfe3 None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:42.496847 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a62ec-572d-41e4-88bd-312a9efadfe3 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:42.497144 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a62ec-572d-41e4-88bd-312a9efadfe3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:42.498791 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a62ec-572d-41e4-88bd-312a9efadfe3 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:42.536966 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:42.545972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 482/965] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 31 10:45:42 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:42.607068 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dcb049-f09c-493b-96db-f17a5753adb9 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:42.607276 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dcb049-f09c-493b-96db-f17a5753adb9 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:42.607559 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dcb049-f09c-493b-96db-f17a5753adb9 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:42.609293 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 484/966] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:42.616131 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ebcb1-d5bc-46f5-b39e-60cce64d6ec5 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:42.616714 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ebcb1-d5bc-46f5-b39e-60cce64d6ec5 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:42.617199 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ebcb1-d5bc-46f5-b39e-60cce64d6ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:42.650914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-dd0ebcb1-d5bc-46f5-b39e-60cce64d6ec5 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:42.653173 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d64d1-cf27-4ed1-9ecf-2fb546ce0168 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:42.653700 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d64d1-cf27-4ed1-9ecf-2fb546ce0168 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:42.654176 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d64d1-cf27-4ed1-9ecf-2fb546ce0168 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:42.654376 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-dd0ebcb1-d5bc-46f5-b39e-60cce64d6ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:42.656422 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 485/967] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:45:42.662672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20852052-470f-4c1d-b168-12d5bc7fa047 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:42.663342 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20852052-470f-4c1d-b168-12d5bc7fa047 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:42.663812 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20852052-470f-4c1d-b168-12d5bc7fa047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:42.684208 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-20852052-470f-4c1d-b168-12d5bc7fa047 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:42.685915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-20852052-470f-4c1d-b168-12d5bc7fa047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:42.774106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 483/968] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:45:42.785344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 486/969] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:45:43.049477 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ebcb1-d5bc-46f5-b39e-60cce64d6ec5 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:43.059518 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20852052-470f-4c1d-b168-12d5bc7fa047 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:43.149517 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:43.163330 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ebcb1-d5bc-46f5-b39e-60cce64d6ec5 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:43.174064 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20852052-470f-4c1d-b168-12d5bc7fa047 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:43.197961 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:43.205871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c5e30-5b60-4d28-ba9f-f925889f4b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:43.206895 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c5e30-5b60-4d28-ba9f-f925889f4b2b None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:43.207111 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c5e30-5b60-4d28-ba9f-f925889f4b2b None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:43.210789 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c5e30-5b60-4d28-ba9f-f925889f4b2b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:43.211568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0b98e-121a-4598-a96a-8abbe92af20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:43.212455 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0b98e-121a-4598-a96a-8abbe92af20b None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:43.212998 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0b98e-121a-4598-a96a-8abbe92af20b None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:43.213557 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0b98e-121a-4598-a96a-8abbe92af20b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:43.214838 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d0b98e-121a-4598-a96a-8abbe92af20b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:43.216477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d0b98e-121a-4598-a96a-8abbe92af20b None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:43.218034 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c5e30-5b60-4d28-ba9f-f925889f4b2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:45:43.222950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-216c5e30-5b60-4d28-ba9f-f925889f4b2b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 10:45:43.222950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:43.222950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:43.222950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:43.222950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:43.222950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:43.222950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:43.222950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:43.222950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:43.222950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:43.222950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:43.222950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:43.222950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:45:43.222950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:45:43.222950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:45:43.222950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:45:43.223901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:45:43.223901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:45:43.223901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:45:43.223901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:45:43.223901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:43.223901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:43.223901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:43.223901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:43.223901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:43.223901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:43.223901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:43.223901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:43.223901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:43.223901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:43.223901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:43.223901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:43.224759 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:43.224759 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:43.224759 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:43.224759 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:43.224759 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:45:43.224759 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:45:43.224759 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:45:43.224759 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:45:43.224759 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:45:43.224759 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:45:43.224759 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 10:45:43.224759 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:43.224759 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 484/970] 10.222.0.16 () {56 vars in 1120 bytes} [Wed Aug 31 10:45:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:43.229083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-f7d0b98e-121a-4598-a96a-8abbe92af20b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:45:43.229083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:43.229083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:45:43.229083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:45:43.229083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:45:43.229083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:45:43.229083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:45:43.229083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:43.229083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:45:43.229083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:43.229083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:45:43.229083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:43.229083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:43.229083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:43.229083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:43.229083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:43.229083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:43.230713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:43.230713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:43.230713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:43.230713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:43.230713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:43.230713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:43.230713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:45:43.230713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:45:43.230713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:45:43.230713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:45:43.230713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:43.230713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:43.230713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:43.230713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:43.230713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:43.230713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:43.231552 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:43.231552 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:43.231552 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:43.231552 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:43.231552 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:43.231552 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:43.231552 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:43.231552 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:43.231552 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:43.231552 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:43.231552 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:45:43.231552 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:43.231552 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:45:43.231552 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:43.232284 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c5e30-5b60-4d28-ba9f-f925889f4b2b None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:43.232364 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 487/971] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Aug 31 10:45:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:43.238219 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0b98e-121a-4598-a96a-8abbe92af20b None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:43.350561 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c5e30-5b60-4d28-ba9f-f925889f4b2b None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:43.352036 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0b98e-121a-4598-a96a-8abbe92af20b None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:43.387787 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ce8ce-ad42-468b-a384-5835fd0be9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:43.388399 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ce8ce-ad42-468b-a384-5835fd0be9cc None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:43.388675 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ce8ce-ad42-468b-a384-5835fd0be9cc None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:43.388946 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ce8ce-ad42-468b-a384-5835fd0be9cc None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:43.389189 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f29ba-1ce1-4bbd-b2d0-3969be021acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:43.389612 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ce8ce-ad42-468b-a384-5835fd0be9cc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:43.389829 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f29ba-1ce1-4bbd-b2d0-3969be021acb None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:43.389925 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ce8ce-ad42-468b-a384-5835fd0be9cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:43.390107 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f29ba-1ce1-4bbd-b2d0-3969be021acb None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:43.390374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f29ba-1ce1-4bbd-b2d0-3969be021acb None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:43.391019 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f29ba-1ce1-4bbd-b2d0-3969be021acb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:43.391314 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f29ba-1ce1-4bbd-b2d0-3969be021acb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:43.391406 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ce8ce-ad42-468b-a384-5835fd0be9cc None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:43.392658 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f29ba-1ce1-4bbd-b2d0-3969be021acb None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:43.397851 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 488/972] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Aug 31 10:45:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:43.404224 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f29ba-1ce1-4bbd-b2d0-3969be021acb None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:43.406963 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 485/973] 10.222.0.16 () {56 vars in 1115 bytes} [Wed Aug 31 10:45:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:43.407122 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:43.411867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ce8ce-ad42-468b-a384-5835fd0be9cc None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:43.470215 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:43.516553 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f29ba-1ce1-4bbd-b2d0-3969be021acb None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:43.521731 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ce8ce-ad42-468b-a384-5835fd0be9cc None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:43.550627 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d8b5f-6125-40f1-8000-619c2e5e03db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:43.551228 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d8b5f-6125-40f1-8000-619c2e5e03db None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:43.551500 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d8b5f-6125-40f1-8000-619c2e5e03db None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:43.551770 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d8b5f-6125-40f1-8000-619c2e5e03db None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:43.552394 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d8b5f-6125-40f1-8000-619c2e5e03db None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:43.552691 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d8b5f-6125-40f1-8000-619c2e5e03db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:43.554048 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d8b5f-6125-40f1-8000-619c2e5e03db None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:43.555536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a887277-f7a3-41c5-a382-7a8637642868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:43.556164 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a887277-f7a3-41c5-a382-7a8637642868 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:43.556434 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a887277-f7a3-41c5-a382-7a8637642868 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:43.556700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a887277-f7a3-41c5-a382-7a8637642868 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:43.560020 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 489/974] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Aug 31 10:45:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:43.561220 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4add15a-aa4a-48f4-9aa9-50e0f68099c5 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:43.561501 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4add15a-aa4a-48f4-9aa9-50e0f68099c5 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:43.561773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4add15a-aa4a-48f4-9aa9-50e0f68099c5 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:43.563157 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 490/975] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:43.565535 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4add15a-aa4a-48f4-9aa9-50e0f68099c5 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:43.570915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a887277-f7a3-41c5-a382-7a8637642868 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:43.572406 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a887277-f7a3-41c5-a382-7a8637642868 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:43.586544 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-5a887277-f7a3-41c5-a382-7a8637642868 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:43.586544 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:43.586544 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:43.586544 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:43.586544 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:43.586544 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:43.586544 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:43.586544 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:43.586544 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:43.586544 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:43.586544 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:43.586544 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:43.586544 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:43.586544 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:43.586544 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:43.586544 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:43.587643 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:43.587643 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:43.587643 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:43.587643 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:43.587643 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:43.587643 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:43.587643 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:43.587643 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:43.587643 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:43.587643 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:43.587643 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:43.587643 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:43.587643 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:43.587643 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:43.587643 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:43.587643 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:43.587643 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:43.588942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:43.588942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:43.588942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:43.588942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 486/976] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:43.588942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d71c9-c1e4-4d2f-8401-35872ec61e25 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:43.589288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d71c9-c1e4-4d2f-8401-35872ec61e25 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:43.589370 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d71c9-c1e4-4d2f-8401-35872ec61e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:43.607364 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-885d71c9-c1e4-4d2f-8401-35872ec61e25 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:43.608478 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-885d71c9-c1e4-4d2f-8401-35872ec61e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:43.637415 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:43.673510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4add15a-aa4a-48f4-9aa9-50e0f68099c5 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:43.676931 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 487/977] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:45:43.677758 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d71c9-c1e4-4d2f-8401-35872ec61e25 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:43.710021 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e0243-afa2-4287-b491-6ca8e12d4e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:43.710904 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e0243-afa2-4287-b491-6ca8e12d4e69 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:43.711464 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e0243-afa2-4287-b491-6ca8e12d4e69 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:43.711594 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e0243-afa2-4287-b491-6ca8e12d4e69 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:43.712327 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830e0243-afa2-4287-b491-6ca8e12d4e69 None admin] RBAC: Adding query filter params (interface=admin, service_id=001af680fba24503affd9ed78b36e68b, region_id=RegionOne) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:43.712642 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830e0243-afa2-4287-b491-6ca8e12d4e69 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:43.715280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830e0243-afa2-4287-b491-6ca8e12d4e69 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:43.720531 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:43.725874 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 491/978] 10.222.0.16 () {56 vars in 1249 bytes} [Wed Aug 31 10:45:43 2022] GET /identity/v3/endpoints?service_id=001af680fba24503affd9ed78b36e68b&interface=admin®ion_id=RegionOne => generated 193 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:43.739436 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e0243-afa2-4287-b491-6ca8e12d4e69 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:43.800309 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d71c9-c1e4-4d2f-8401-35872ec61e25 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:43.832871 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e0243-afa2-4287-b491-6ca8e12d4e69 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:43.837741 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb65271-b198-4313-8e40-4a48929d81da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:43.838346 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb65271-b198-4313-8e40-4a48929d81da None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:43.838657 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb65271-b198-4313-8e40-4a48929d81da None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:43.839567 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb65271-b198-4313-8e40-4a48929d81da None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:43.840208 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb65271-b198-4313-8e40-4a48929d81da None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:43.840507 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb65271-b198-4313-8e40-4a48929d81da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:43.841904 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb65271-b198-4313-8e40-4a48929d81da None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:43.854313 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 488/979] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:43.860584 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb65271-b198-4313-8e40-4a48929d81da None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:43.869308 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:43.876196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47beadf7-d96c-4498-a24e-8e49b8e0dcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:43.876805 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47beadf7-d96c-4498-a24e-8e49b8e0dcbe None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:43.877080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47beadf7-d96c-4498-a24e-8e49b8e0dcbe None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:43.877351 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47beadf7-d96c-4498-a24e-8e49b8e0dcbe None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:43.877958 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47beadf7-d96c-4498-a24e-8e49b8e0dcbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:43.879350 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47beadf7-d96c-4498-a24e-8e49b8e0dcbe None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:43.885258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 492/980] 10.222.0.16 () {56 vars in 1087 bytes} [Wed Aug 31 10:45:43 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:45:43.958187 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47beadf7-d96c-4498-a24e-8e49b8e0dcbe None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:43.967203 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:43.975227 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb65271-b198-4313-8e40-4a48929d81da None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:44.016288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591e9f6-0529-4ea1-8010-36e1ae122b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:44.016934 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591e9f6-0529-4ea1-8010-36e1ae122b58 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:44.017214 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591e9f6-0529-4ea1-8010-36e1ae122b58 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:44.017488 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591e9f6-0529-4ea1-8010-36e1ae122b58 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:44.025506 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5591e9f6-0529-4ea1-8010-36e1ae122b58 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:44.027829 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5591e9f6-0529-4ea1-8010-36e1ae122b58 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:44.041534 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-5591e9f6-0529-4ea1-8010-36e1ae122b58 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 10:45:44.041534 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:44.041534 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:44.041534 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:44.041534 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:44.041534 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:44.041534 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:44.041534 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:44.041534 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:44.041534 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:44.041534 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:44.041534 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:44.041534 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:45:44.041534 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:45:44.041534 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:45:44.041534 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:45:44.043257 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:44.043257 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:44.043257 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:45:44.043257 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:44.043257 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:45:44.043257 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:44.043257 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:44.043257 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:44.043257 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:44.043257 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:44.043257 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:44.043257 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:44.043257 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:44.043257 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:44.043257 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:44.043257 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:44.043257 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:44.043257 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:44.044409 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:44.044409 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:44.044409 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:45:44.044409 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:45:44.044409 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:45:44.044409 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:45:44.044409 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:45:44.044409 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:45:44.044409 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 10:45:44.044409 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:44.044409 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 489/981] 10.222.0.16 () {56 vars in 1117 bytes} [Wed Aug 31 10:45:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:44.049980 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591e9f6-0529-4ea1-8010-36e1ae122b58 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:44.079203 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47beadf7-d96c-4498-a24e-8e49b8e0dcbe None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:44.121356 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335668a9-ee90-4107-8346-a7504c37cf80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:44.121800 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335668a9-ee90-4107-8346-a7504c37cf80 None admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:44.122075 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335668a9-ee90-4107-8346-a7504c37cf80 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:44.122346 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335668a9-ee90-4107-8346-a7504c37cf80 None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:44.123028 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335668a9-ee90-4107-8346-a7504c37cf80 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:44.124383 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335668a9-ee90-4107-8346-a7504c37cf80 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:44.164439 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591e9f6-0529-4ea1-8010-36e1ae122b58 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:44.198697 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be25750-e97b-4c22-a008-e071672d0747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:44.199648 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be25750-e97b-4c22-a008-e071672d0747 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:44.199932 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be25750-e97b-4c22-a008-e071672d0747 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:44.200208 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be25750-e97b-4c22-a008-e071672d0747 None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:44.200942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be25750-e97b-4c22-a008-e071672d0747 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:44.201249 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be25750-e97b-4c22-a008-e071672d0747 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:44.202691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be25750-e97b-4c22-a008-e071672d0747 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:44.220835 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 490/982] 10.222.0.16 () {56 vars in 1149 bytes} [Wed Aug 31 10:45:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:44.226495 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 493/983] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Aug 31 10:45:43 2022] POST /identity/v3/services => generated 242 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 10:45:44.227010 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be25750-e97b-4c22-a008-e071672d0747 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:44.248843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:44.341212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be25750-e97b-4c22-a008-e071672d0747 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:44.388034 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244ace7-62bf-46e2-8f63-312a553feb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:44.388034 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244ace7-62bf-46e2-8f63-312a553feb29 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:44.388712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244ace7-62bf-46e2-8f63-312a553feb29 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:44.388712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244ace7-62bf-46e2-8f63-312a553feb29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:44.458981 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244ace7-62bf-46e2-8f63-312a553feb29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:44.460460 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244ace7-62bf-46e2-8f63-312a553feb29 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:44.471358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:44.563998 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 491/984] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:45:44 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 10:45:44.569467 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335668a9-ee90-4107-8346-a7504c37cf80 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:44.627155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:44.675508 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335668a9-ee90-4107-8346-a7504c37cf80 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:44.708878 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83b2c9-25b8-45e8-9135-c03b6636de32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:44.709891 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83b2c9-25b8-45e8-9135-c03b6636de32 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:44.710487 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83b2c9-25b8-45e8-9135-c03b6636de32 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:44.710729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83b2c9-25b8-45e8-9135-c03b6636de32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:44.723004 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83b2c9-25b8-45e8-9135-c03b6636de32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:44.724618 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83b2c9-25b8-45e8-9135-c03b6636de32 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:44.740888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-ce83b2c9-25b8-45e8-9135-c03b6636de32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:44.740888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:44.740888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:44.740888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:44.740888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:44.740888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:44.740888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:44.740888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:44.740888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:44.740888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:44.740888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:44.740888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:44.740888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:44.740888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:44.740888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:44.740888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:44.742369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:44.742369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:44.742369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:44.742369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:44.742369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:44.742369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:44.742369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:44.742369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:44.742369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:44.742369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:44.742369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:44.742369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:44.742369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:44.742369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:44.742369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:44.742369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:44.742369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:44.743521 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:44.743521 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:44.743521 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:44.743521 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 494/985] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:44.749882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244ace7-62bf-46e2-8f63-312a553feb29 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:44.853615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244ace7-62bf-46e2-8f63-312a553feb29 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:44.864545 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:44.890582 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ab5c9-c242-49d2-aabc-3d2731ea7c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:44.891194 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ab5c9-c242-49d2-aabc-3d2731ea7c90 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:44.891483 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ab5c9-c242-49d2-aabc-3d2731ea7c90 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:44.891754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ab5c9-c242-49d2-aabc-3d2731ea7c90 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:44.892407 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ab5c9-c242-49d2-aabc-3d2731ea7c90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:44.892710 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ab5c9-c242-49d2-aabc-3d2731ea7c90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:44.894096 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ab5c9-c242-49d2-aabc-3d2731ea7c90 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:44.906772 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 492/986] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:44.912158 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83b2c9-25b8-45e8-9135-c03b6636de32 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:45.017768 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83b2c9-25b8-45e8-9135-c03b6636de32 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:45.024983 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:45.053023 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec242cd4-e055-423c-95f7-a3946447e938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:45.053654 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec242cd4-e055-423c-95f7-a3946447e938 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:45.053934 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec242cd4-e055-423c-95f7-a3946447e938 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:45.054207 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec242cd4-e055-423c-95f7-a3946447e938 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:45.061043 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec242cd4-e055-423c-95f7-a3946447e938 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:45.062515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec242cd4-e055-423c-95f7-a3946447e938 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:45.074663 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-ec242cd4-e055-423c-95f7-a3946447e938 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:45.074663 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:45.074663 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:45.074663 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:45.074663 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:45.074663 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:45.074663 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:45.074663 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:45.074663 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:45.074663 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:45.074663 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:45.074663 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:45.074663 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:45:45.074663 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:45:45.074663 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:45:45.075549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:45:45.075549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:45.075549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:45.075549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:45.075549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:45.075549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:45.075549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:45.075549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:45.075549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:45.075549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:45.075549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:45.075549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:45.075549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:45.075549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:45.075549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:45.075549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:45.075549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:45.076447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:45:45.076447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:45:45.076447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:45:45.076447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:45:45.076447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:45:45.076447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:45:45.076447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:45.076447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:45.078711 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 495/987] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:45:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:45.081648 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ab5c9-c242-49d2-aabc-3d2731ea7c90 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:45.193154 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ab5c9-c242-49d2-aabc-3d2731ea7c90 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:45.239673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa3a78d-bced-4319-8afb-e1c2c13f685a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:45.240701 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3a78d-bced-4319-8afb-e1c2c13f685a None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:45.241195 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3a78d-bced-4319-8afb-e1c2c13f685a None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:45.241678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3a78d-bced-4319-8afb-e1c2c13f685a None admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:45.242975 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa3a78d-bced-4319-8afb-e1c2c13f685a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:45.243521 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa3a78d-bced-4319-8afb-e1c2c13f685a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:45.246992 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa3a78d-bced-4319-8afb-e1c2c13f685a None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:45.271826 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 493/988] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:45:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:45.279375 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec242cd4-e055-423c-95f7-a3946447e938 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:45.390636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec242cd4-e055-423c-95f7-a3946447e938 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:45.424120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea58f0d-8459-413a-ab5d-84b5927991f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:45.424720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea58f0d-8459-413a-ab5d-84b5927991f5 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:45.424994 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea58f0d-8459-413a-ab5d-84b5927991f5 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:45.425260 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea58f0d-8459-413a-ab5d-84b5927991f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:45.425975 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea58f0d-8459-413a-ab5d-84b5927991f5 None admin] RBAC: Adding query filter params (role.id=1715f4cc50354636b1f7c3a9864e1e80, scope.project.id=8db9b3f8a9f041938ac7b39003a35015, user.id=7796ebff70054de090e02c58bf7000f4) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:45.426276 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea58f0d-8459-413a-ab5d-84b5927991f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:45.427725 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea58f0d-8459-413a-ab5d-84b5927991f5 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:45.434133 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 496/989] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:45:45 2022] GET /identity/v3/role_assignments?user.id=7796ebff70054de090e02c58bf7000f4&scope.project.id=8db9b3f8a9f041938ac7b39003a35015&role.id=1715f4cc50354636b1f7c3a9864e1e80 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:45.838091 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727aeb55-353a-42a8-a126-cc3e35126939 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:45.839317 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727aeb55-353a-42a8-a126-cc3e35126939 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:45.839317 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727aeb55-353a-42a8-a126-cc3e35126939 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:45.840748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 494/990] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:45.845796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c62a2-120a-4ff6-9044-72fa604c2d3b None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:45.846082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c62a2-120a-4ff6-9044-72fa604c2d3b None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:45.846351 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c62a2-120a-4ff6-9044-72fa604c2d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:45.865544 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-5c4c62a2-120a-4ff6-9044-72fa604c2d3b None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:45.866787 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-5c4c62a2-120a-4ff6-9044-72fa604c2d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:45.892505 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:45.945735 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 497/991] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:45 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:45:46.183088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef31a8d-5d9e-42a6-9472-badee1c46176 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:46.183443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef31a8d-5d9e-42a6-9472-badee1c46176 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:46.183813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef31a8d-5d9e-42a6-9472-badee1c46176 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:46.185331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 495/992] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:45:46.190238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab4fa5-aac8-47e0-aba5-063a24d64237 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:46.190566 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab4fa5-aac8-47e0-aba5-063a24d64237 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:46.190838 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab4fa5-aac8-47e0-aba5-063a24d64237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:46.209815 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-08ab4fa5-aac8-47e0-aba5-063a24d64237 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:46.211037 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-08ab4fa5-aac8-47e0-aba5-063a24d64237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:46.221154 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef31a8d-5d9e-42a6-9472-badee1c46176 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:46.249988 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:46.294777 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 498/993] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:46 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:45:46.332757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef31a8d-5d9e-42a6-9472-badee1c46176 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:46.368396 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cedc8c-c503-431c-ac9b-391675b10f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:46.369574 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cedc8c-c503-431c-ac9b-391675b10f12 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:46.370747 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cedc8c-c503-431c-ac9b-391675b10f12 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:46.371193 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cedc8c-c503-431c-ac9b-391675b10f12 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:46.371935 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cedc8c-c503-431c-ac9b-391675b10f12 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:46.373562 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cedc8c-c503-431c-ac9b-391675b10f12 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:46.382664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-85cedc8c-c503-431c-ac9b-391675b10f12 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:45:46.382664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:46.382664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:45:46.382664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:45:46.382664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:45:46.382664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:45:46.382664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:45:46.382664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:46.382664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:45:46.382664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:46.382664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:45:46.382664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:46.382664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:46.382664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:46.382664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:46.382664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:46.382664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:46.383615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:46.383615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:46.383615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:46.383615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:46.383615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:46.383615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:46.383615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:45:46.383615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:45:46.383615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:45:46.383615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:45:46.383615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:46.383615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:46.383615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:46.383615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:46.383615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:46.383615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:46.384443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:46.384443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:46.384443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:46.384443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:46.384443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:46.384443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:46.384443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:46.384443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:46.384443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:46.384443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:46.384443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:45:46.384443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:46.384443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:45:46.384443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:46.386875 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 496/994] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Aug 31 10:45:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:46.392461 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab4fa5-aac8-47e0-aba5-063a24d64237 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:46.489819 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:46.500349 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab4fa5-aac8-47e0-aba5-063a24d64237 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:46.534702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce792e1-1a1d-4259-a590-c45d7d8fddd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:46.535319 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce792e1-1a1d-4259-a590-c45d7d8fddd1 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:46.535599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce792e1-1a1d-4259-a590-c45d7d8fddd1 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:46.535872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce792e1-1a1d-4259-a590-c45d7d8fddd1 None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:46.536612 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce792e1-1a1d-4259-a590-c45d7d8fddd1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:46.536918 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce792e1-1a1d-4259-a590-c45d7d8fddd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:46.538514 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce792e1-1a1d-4259-a590-c45d7d8fddd1 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:46.544344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 499/995] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Aug 31 10:45:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:46.550070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cedc8c-c503-431c-ac9b-391675b10f12 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:46.566208 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce792e1-1a1d-4259-a590-c45d7d8fddd1 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:46.649154 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:46.658318 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cedc8c-c503-431c-ac9b-391675b10f12 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:46.673335 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce792e1-1a1d-4259-a590-c45d7d8fddd1 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:46.696624 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195a581-0910-4d2c-97c4-254b50f383a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:46.697288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195a581-0910-4d2c-97c4-254b50f383a0 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:46.697512 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195a581-0910-4d2c-97c4-254b50f383a0 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:46.697788 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195a581-0910-4d2c-97c4-254b50f383a0 None admin] PATH_INFO: `/v3/services` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:46.698558 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0195a581-0910-4d2c-97c4-254b50f383a0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:46.698862 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0195a581-0910-4d2c-97c4-254b50f383a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:46.700228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0195a581-0910-4d2c-97c4-254b50f383a0 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:46.705513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 497/996] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Aug 31 10:45:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:46.708100 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a7761-199f-48b6-918f-a4f4a6a2a667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:46.708724 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a7761-199f-48b6-918f-a4f4a6a2a667 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:46.708998 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a7761-199f-48b6-918f-a4f4a6a2a667 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:46.709264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a7761-199f-48b6-918f-a4f4a6a2a667 None admin] PATH_INFO: `/v3/services/compute` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:46.709837 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a7761-199f-48b6-918f-a4f4a6a2a667 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:46.711117 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195a581-0910-4d2c-97c4-254b50f383a0 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:46.711229 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a7761-199f-48b6-918f-a4f4a6a2a667 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:46.721729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-b01a7761-199f-48b6-918f-a4f4a6a2a667 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 10:45:46.721729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:46.721729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:45:46.721729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:45:46.721729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:45:46.721729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:45:46.721729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:45:46.721729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:46.721729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 10:45:46.721729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:46.721729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:45:46.721729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:46.721729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:46.721729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:46.721729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:46.721729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:46.721729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:46.722976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:46.722976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:46.722976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:46.722976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:46.722976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:46.722976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:46.722976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:45:46.722976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:45:46.722976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:45:46.722976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:45:46.722976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:46.722976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:46.722976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:46.722976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:46.722976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:46.722976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:46.723928 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:46.723928 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:46.723928 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:46.723928 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:46.723928 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:46.723928 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:46.723928 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:46.723928 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:46.723928 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:46.723928 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:46.723928 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:45:46.723928 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:46.723928 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 10:45:46.723928 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:46.725420 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 500/997] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:45:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:46.728768 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a7761-199f-48b6-918f-a4f4a6a2a667 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:46.748665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:46.819008 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195a581-0910-4d2c-97c4-254b50f383a0 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:46.836392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a7761-199f-48b6-918f-a4f4a6a2a667 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:46.856204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52bcca-03ba-4c36-a72e-c95b04e110ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:46.856896 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52bcca-03ba-4c36-a72e-c95b04e110ac None admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:46.857097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52bcca-03ba-4c36-a72e-c95b04e110ac None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:46.857368 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52bcca-03ba-4c36-a72e-c95b04e110ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:46.858035 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf52bcca-03ba-4c36-a72e-c95b04e110ac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:46.859447 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf52bcca-03ba-4c36-a72e-c95b04e110ac None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:46.872678 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc44fa4-a76a-4617-b4e9-4e3bf15fd815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:46.873275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc44fa4-a76a-4617-b4e9-4e3bf15fd815 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:46.878508 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc44fa4-a76a-4617-b4e9-4e3bf15fd815 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:46.878508 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc44fa4-a76a-4617-b4e9-4e3bf15fd815 None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:46.878508 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc44fa4-a76a-4617-b4e9-4e3bf15fd815 None admin] RBAC: Adding query filter params (name=compute) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:46.878508 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc44fa4-a76a-4617-b4e9-4e3bf15fd815 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:46.878508 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc44fa4-a76a-4617-b4e9-4e3bf15fd815 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:46.880798 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 501/998] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Aug 31 10:45:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:46.886466 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc44fa4-a76a-4617-b4e9-4e3bf15fd815 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:46.939192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 498/999] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 10:45:46 2022] POST /identity/v3/endpoints => generated 325 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 10:45:46.978940 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:47.000731 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc44fa4-a76a-4617-b4e9-4e3bf15fd815 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:47.046417 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468df65-c6ef-4339-8851-efea89687d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:47.047059 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468df65-c6ef-4339-8851-efea89687d50 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:47.047336 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468df65-c6ef-4339-8851-efea89687d50 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:47.047605 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468df65-c6ef-4339-8851-efea89687d50 None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:47.048240 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6468df65-c6ef-4339-8851-efea89687d50 None admin] RBAC: Adding query filter params (type=compute) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:47.048547 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6468df65-c6ef-4339-8851-efea89687d50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:47.049941 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6468df65-c6ef-4339-8851-efea89687d50 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:47.061034 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 502/1000] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Aug 31 10:45:46 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:47.063119 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52bcca-03ba-4c36-a72e-c95b04e110ac None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:47.084353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:47.177958 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52bcca-03ba-4c36-a72e-c95b04e110ac None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:47.213653 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fec0d3-9f65-4c2b-a0d5-73ad88ff1f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:47.214209 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fec0d3-9f65-4c2b-a0d5-73ad88ff1f00 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:47.214516 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fec0d3-9f65-4c2b-a0d5-73ad88ff1f00 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:47.214791 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fec0d3-9f65-4c2b-a0d5-73ad88ff1f00 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:47.215496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fec0d3-9f65-4c2b-a0d5-73ad88ff1f00 None admin] RBAC: Adding query filter params (interface=public, service_id=359092d715624c21b64a9d482a24c409, region_id=RegionOne) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:47.215802 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fec0d3-9f65-4c2b-a0d5-73ad88ff1f00 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:47.217196 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fec0d3-9f65-4c2b-a0d5-73ad88ff1f00 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:47.226925 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 499/1001] 10.222.0.16 () {56 vars in 1251 bytes} [Wed Aug 31 10:45:47 2022] GET /identity/v3/endpoints?service_id=359092d715624c21b64a9d482a24c409&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:47.228465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468df65-c6ef-4339-8851-efea89687d50 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:47.398352 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468df65-c6ef-4339-8851-efea89687d50 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:47.419538 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe4499-b4b1-44f6-8e5a-b6ad50120e8b None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:47.419859 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe4499-b4b1-44f6-8e5a-b6ad50120e8b None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:47.420563 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe4499-b4b1-44f6-8e5a-b6ad50120e8b None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:47.423295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 500/1002] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:45:47.426901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10d2d4-8714-4fbf-a843-08c1726eb16e None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:47.427243 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10d2d4-8714-4fbf-a843-08c1726eb16e None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:47.427582 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10d2d4-8714-4fbf-a843-08c1726eb16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:47.445025 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9555875f-2803-4590-81a2-3743e20c5fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:47.445636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9555875f-2803-4590-81a2-3743e20c5fd4 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:47.445912 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9555875f-2803-4590-81a2-3743e20c5fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:47.446181 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9555875f-2803-4590-81a2-3743e20c5fd4 None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:47.446826 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9555875f-2803-4590-81a2-3743e20c5fd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:47.448692 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9555875f-2803-4590-81a2-3743e20c5fd4 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:47.455401 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 503/1003] 10.222.0.16 () {56 vars in 1087 bytes} [Wed Aug 31 10:45:47 2022] GET /identity/v3/services => generated 1727 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:45:47.455546 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-bf10d2d4-8714-4fbf-a843-08c1726eb16e None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:47.456591 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-bf10d2d4-8714-4fbf-a843-08c1726eb16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:47.538499 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 501/1004] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:47 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:45:47.818098 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9555875f-2803-4590-81a2-3743e20c5fd4 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:47.828745 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:47.923793 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9555875f-2803-4590-81a2-3743e20c5fd4 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:47.960522 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda9df4-4d91-4e2f-8889-5638d8f786ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:47.961158 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda9df4-4d91-4e2f-8889-5638d8f786ed None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:47.961430 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda9df4-4d91-4e2f-8889-5638d8f786ed None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:47.961696 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda9df4-4d91-4e2f-8889-5638d8f786ed None admin] PATH_INFO: `/v3/services/image` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:47.962279 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda9df4-4d91-4e2f-8889-5638d8f786ed None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:47.963732 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda9df4-4d91-4e2f-8889-5638d8f786ed None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:47.973332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-edda9df4-4d91-4e2f-8889-5638d8f786ed None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 10:45:47.973332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:47.973332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:45:47.973332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:45:47.973332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:45:47.973332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:45:47.973332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:45:47.973332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:47.973332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 10:45:47.973332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:47.973332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:45:47.973332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:47.973332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:47.973332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:47.973332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:47.973332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:47.973332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:47.974341 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:47.974341 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:47.974341 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:47.974341 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:47.974341 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:47.974341 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:47.974341 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:45:47.974341 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:45:47.974341 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:45:47.974341 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:45:47.974341 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:47.974341 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:47.974341 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:47.974341 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:47.974341 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:47.974341 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:47.979235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:47.979235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:47.979235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:47.979235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:47.979235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:47.979235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:47.979235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:47.979235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:47.979235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:47.979235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:47.979235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:45:47.979235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:47.979235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 10:45:47.979235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:47.979235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 504/1005] 10.222.0.16 () {56 vars in 1105 bytes} [Wed Aug 31 10:45:47 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:47.980954 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10d2d4-8714-4fbf-a843-08c1726eb16e None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:47.993115 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:48.088461 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10d2d4-8714-4fbf-a843-08c1726eb16e None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:48.131912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b7f8d-2fcc-4c88-82a3-a218cbb99089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:48.132522 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b7f8d-2fcc-4c88-82a3-a218cbb99089 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:48.132796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b7f8d-2fcc-4c88-82a3-a218cbb99089 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:48.133088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b7f8d-2fcc-4c88-82a3-a218cbb99089 None admin] PATH_INFO: `/v3/services` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:48.133710 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b7f8d-2fcc-4c88-82a3-a218cbb99089 None admin] RBAC: Adding query filter params (name=image) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:48.134004 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b7f8d-2fcc-4c88-82a3-a218cbb99089 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:48.135389 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b7f8d-2fcc-4c88-82a3-a218cbb99089 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:48.145077 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 502/1006] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:48.146685 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda9df4-4d91-4e2f-8889-5638d8f786ed None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:48.218709 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:48.249547 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda9df4-4d91-4e2f-8889-5638d8f786ed None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:48.284246 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd4a99-4529-4d15-9609-1269439aa789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:48.284852 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd4a99-4529-4d15-9609-1269439aa789 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:48.285122 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd4a99-4529-4d15-9609-1269439aa789 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:48.285389 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd4a99-4529-4d15-9609-1269439aa789 None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:48.286010 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd4a99-4529-4d15-9609-1269439aa789 None admin] RBAC: Adding query filter params (type=image) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:48.286359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd4a99-4529-4d15-9609-1269439aa789 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:48.287787 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd4a99-4529-4d15-9609-1269439aa789 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:48.292712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 505/1007] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:48 2022] GET /identity/v3/services?type=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:48.944258 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626a6c7-cb06-4721-a22c-3e481ca87818 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:48.944666 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626a6c7-cb06-4721-a22c-3e481ca87818 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:48.944837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626a6c7-cb06-4721-a22c-3e481ca87818 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:48.946290 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 503/1008] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:48.951812 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73716f7a-1834-440c-a3f2-2b07db851197 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:48.952259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73716f7a-1834-440c-a3f2-2b07db851197 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:48.952423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73716f7a-1834-440c-a3f2-2b07db851197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:48.971729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-73716f7a-1834-440c-a3f2-2b07db851197 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:48.972930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-73716f7a-1834-440c-a3f2-2b07db851197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:49.043701 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 506/1009] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:48 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:45:49.318422 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626a6c7-cb06-4721-a22c-3e481ca87818 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:49.402897 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:49.407456 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7f861-3c1a-46f7-9ed5-c401c66e7639 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:49.407738 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7f861-3c1a-46f7-9ed5-c401c66e7639 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:49.408011 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7f861-3c1a-46f7-9ed5-c401c66e7639 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:49.409398 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 507/1010] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:45:49.414848 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9723ea-afb8-470c-b67d-742ddf868164 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:49.415346 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9723ea-afb8-470c-b67d-742ddf868164 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:49.415558 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9723ea-afb8-470c-b67d-742ddf868164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:49.429514 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626a6c7-cb06-4721-a22c-3e481ca87818 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:49.434065 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-4f9723ea-afb8-470c-b67d-742ddf868164 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:49.435251 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-4f9723ea-afb8-470c-b67d-742ddf868164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:49.469858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0acf8-e6b2-4dd3-9ffa-76c6cebac537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:49.470423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0acf8-e6b2-4dd3-9ffa-76c6cebac537 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:49.470732 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0acf8-e6b2-4dd3-9ffa-76c6cebac537 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:49.470991 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0acf8-e6b2-4dd3-9ffa-76c6cebac537 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:49.472117 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0acf8-e6b2-4dd3-9ffa-76c6cebac537 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:49.473934 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0acf8-e6b2-4dd3-9ffa-76c6cebac537 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:49.475338 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 504/1011] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:49.483586 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0acf8-e6b2-4dd3-9ffa-76c6cebac537 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:49.503092 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:49.513514 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 508/1012] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:49 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:45:49.591498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0acf8-e6b2-4dd3-9ffa-76c6cebac537 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:49.631937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4286b-6cd8-4372-9d81-15ff285bc722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:49.632548 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4286b-6cd8-4372-9d81-15ff285bc722 None admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:49.632824 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4286b-6cd8-4372-9d81-15ff285bc722 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:49.633095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4286b-6cd8-4372-9d81-15ff285bc722 None admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:49.633774 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca4286b-6cd8-4372-9d81-15ff285bc722 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:49.635202 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca4286b-6cd8-4372-9d81-15ff285bc722 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:49.651859 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:49.738415 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 505/1013] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Aug 31 10:45:49 2022] POST /identity/v3/projects => generated 311 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:45:49.792679 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9723ea-afb8-470c-b67d-742ddf868164 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:49.904280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9723ea-afb8-470c-b67d-742ddf868164 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:49.943532 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29480415-ff58-4541-86dd-c9acb15be9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:49.944158 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29480415-ff58-4541-86dd-c9acb15be9b4 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:49.944429 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29480415-ff58-4541-86dd-c9acb15be9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:49.944689 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29480415-ff58-4541-86dd-c9acb15be9b4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:49.945271 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29480415-ff58-4541-86dd-c9acb15be9b4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:49.946674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29480415-ff58-4541-86dd-c9acb15be9b4 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:49.956252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-29480415-ff58-4541-86dd-c9acb15be9b4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 10:45:49.956252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:49.956252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:45:49.956252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:45:49.956252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:45:49.956252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:45:49.956252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:45:49.956252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:49.956252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 10:45:49.956252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:49.956252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:45:49.956252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:49.956252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:49.956252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:49.956252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:49.956252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:49.956252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:49.957644 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:49.957644 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:49.957644 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:49.957644 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:49.957644 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:49.957644 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:49.957644 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:45:49.957644 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:45:49.957644 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:45:49.957644 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:45:49.957644 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:49.957644 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:49.957644 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:49.957644 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:49.957644 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:49.957644 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:49.958494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:49.958494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:49.958494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:49.958494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:49.958494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:49.958494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:49.958494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:49.958494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:49.958494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:49.958494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:49.958494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:45:49.958494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:49.958494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 10:45:49.958494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:49.958494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 509/1014] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:45:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:49.963544 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4286b-6cd8-4372-9d81-15ff285bc722 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:50.067742 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4286b-6cd8-4372-9d81-15ff285bc722 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:50.112794 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff67a0-6a12-4925-9e3e-4ba298a7e0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:50.113417 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff67a0-6a12-4925-9e3e-4ba298a7e0c2 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:50.113694 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff67a0-6a12-4925-9e3e-4ba298a7e0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:50.113962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff67a0-6a12-4925-9e3e-4ba298a7e0c2 None admin] PATH_INFO: `/v3/services` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:50.114724 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff67a0-6a12-4925-9e3e-4ba298a7e0c2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:50.115057 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff67a0-6a12-4925-9e3e-4ba298a7e0c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:50.116427 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff67a0-6a12-4925-9e3e-4ba298a7e0c2 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:50.123165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 506/1015] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Aug 31 10:45:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:50.128861 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29480415-ff58-4541-86dd-c9acb15be9b4 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:50.191977 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:50.217567 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5d8fd-20e6-4f78-b6dc-9e6779751672 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:50.217896 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5d8fd-20e6-4f78-b6dc-9e6779751672 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:50.218123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5d8fd-20e6-4f78-b6dc-9e6779751672 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:50.219550 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 507/1016] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:45:50.224654 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437d6a4-8446-4d70-b01d-6c41a185722b None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:50.224941 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437d6a4-8446-4d70-b01d-6c41a185722b None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:50.225216 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437d6a4-8446-4d70-b01d-6c41a185722b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:50.240525 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29480415-ff58-4541-86dd-c9acb15be9b4 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:50.243528 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-4437d6a4-8446-4d70-b01d-6c41a185722b None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:50.245011 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-4437d6a4-8446-4d70-b01d-6c41a185722b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:50.279218 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d8845-7f5c-499c-9a45-6664e8474262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:50.279830 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d8845-7f5c-499c-9a45-6664e8474262 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:50.280111 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d8845-7f5c-499c-9a45-6664e8474262 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:50.280369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d8845-7f5c-499c-9a45-6664e8474262 None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:50.281012 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d8845-7f5c-499c-9a45-6664e8474262 None admin] RBAC: Adding query filter params (type=compute) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:50.281302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d8845-7f5c-499c-9a45-6664e8474262 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:50.282701 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d8845-7f5c-499c-9a45-6664e8474262 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:50.292902 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 510/1017] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Aug 31 10:45:50 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:50.293959 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d8845-7f5c-499c-9a45-6664e8474262 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:50.314364 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:50.330726 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 508/1018] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:50 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:45:50.406332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d8845-7f5c-499c-9a45-6664e8474262 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:50.430123 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:50.441919 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00808844-f9de-4a7e-bcb4-c76c189a9807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:50.442610 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00808844-f9de-4a7e-bcb4-c76c189a9807 None admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:50.442835 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00808844-f9de-4a7e-bcb4-c76c189a9807 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:50.443113 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00808844-f9de-4a7e-bcb4-c76c189a9807 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:50.443752 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00808844-f9de-4a7e-bcb4-c76c189a9807 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:50.445151 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00808844-f9de-4a7e-bcb4-c76c189a9807 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:50.574530 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 511/1019] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 10:45:50 2022] POST /identity/v3/endpoints => generated 334 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 10:45:50.603942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4437d6a4-8446-4d70-b01d-6c41a185722b None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:50.715500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4437d6a4-8446-4d70-b01d-6c41a185722b None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:50.757350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca159d-b9ab-4d10-a302-c7786e4351b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:50.757981 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca159d-b9ab-4d10-a302-c7786e4351b7 None admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:50.758257 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca159d-b9ab-4d10-a302-c7786e4351b7 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:50.758552 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca159d-b9ab-4d10-a302-c7786e4351b7 None admin] PATH_INFO: `/v3/services` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:50.759204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca159d-b9ab-4d10-a302-c7786e4351b7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:50.760681 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca159d-b9ab-4d10-a302-c7786e4351b7 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:50.872684 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 509/1020] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Aug 31 10:45:50 2022] POST /identity/v3/services => generated 242 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:45:51.718960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f69192-ad09-4e8a-806a-05aeb3a7fa88 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:51.719481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f69192-ad09-4e8a-806a-05aeb3a7fa88 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:51.719481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f69192-ad09-4e8a-806a-05aeb3a7fa88 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:51.721166 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 512/1021] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:51.726464 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd3253-eb1f-4b61-89f0-0497fef0a90e None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:51.726885 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd3253-eb1f-4b61-89f0-0497fef0a90e None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:51.727054 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd3253-eb1f-4b61-89f0-0497fef0a90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:51.746377 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-34bd3253-eb1f-4b61-89f0-0497fef0a90e None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:51.747621 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-34bd3253-eb1f-4b61-89f0-0497fef0a90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:51.792916 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:51.828666 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 510/1022] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:51 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:45:52.098498 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f69192-ad09-4e8a-806a-05aeb3a7fa88 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:52.204481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f69192-ad09-4e8a-806a-05aeb3a7fa88 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:52.244117 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b724a3-3430-4069-81eb-7a7b01c67bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:52.246170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b724a3-3430-4069-81eb-7a7b01c67bc3 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:52.246484 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b724a3-3430-4069-81eb-7a7b01c67bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:52.246767 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b724a3-3430-4069-81eb-7a7b01c67bc3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:52.247959 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b724a3-3430-4069-81eb-7a7b01c67bc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:52.249409 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b724a3-3430-4069-81eb-7a7b01c67bc3 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:52.251324 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 513/1023] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:52.257019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bd3253-eb1f-4b61-89f0-0497fef0a90e None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:52.363619 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bd3253-eb1f-4b61-89f0-0497fef0a90e None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:52.399130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee2716-57d5-4db5-b705-29c7a12e74bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:52.399819 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee2716-57d5-4db5-b705-29c7a12e74bf None admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:52.400100 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee2716-57d5-4db5-b705-29c7a12e74bf None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:52.400374 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee2716-57d5-4db5-b705-29c7a12e74bf None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:52.401671 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee2716-57d5-4db5-b705-29c7a12e74bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:52.403140 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee2716-57d5-4db5-b705-29c7a12e74bf None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:52.532580 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 511/1024] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 31 10:45:52 2022] POST /identity/v3/users => generated 285 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:45:52.606701 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff48fdd-7eb5-4c4f-82dd-79a5c0f0827f None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:52.607037 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff48fdd-7eb5-4c4f-82dd-79a5c0f0827f None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:52.607251 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff48fdd-7eb5-4c4f-82dd-79a5c0f0827f None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:52.608833 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 514/1025] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:52.614358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec19e6-67e7-466a-839f-829af83efad4 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:52.614624 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec19e6-67e7-466a-839f-829af83efad4 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:52.614876 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec19e6-67e7-466a-839f-829af83efad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:52.641135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-58ec19e6-67e7-466a-839f-829af83efad4 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:52.642356 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-58ec19e6-67e7-466a-839f-829af83efad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:52.666712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:52.723688 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 512/1026] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:45:52.784681 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c668c15-2ab1-4d4d-8436-a8789303dbf5 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:52.785197 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c668c15-2ab1-4d4d-8436-a8789303dbf5 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:52.785197 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c668c15-2ab1-4d4d-8436-a8789303dbf5 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:52.786793 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 515/1027] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:52.792806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba06626-2df4-447c-b9a6-6f070fa6bd2f None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:52.793230 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba06626-2df4-447c-b9a6-6f070fa6bd2f None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:52.793464 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba06626-2df4-447c-b9a6-6f070fa6bd2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:52.813294 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-3ba06626-2df4-447c-b9a6-6f070fa6bd2f None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:52.814514 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-3ba06626-2df4-447c-b9a6-6f070fa6bd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:52.888134 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 513/1028] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:45:52.989103 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c668c15-2ab1-4d4d-8436-a8789303dbf5 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:53.039047 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:53.097404 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c668c15-2ab1-4d4d-8436-a8789303dbf5 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:53.135446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fefadf7-44ba-4a4f-a68c-df5f280be68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:53.136172 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefadf7-44ba-4a4f-a68c-df5f280be68d None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:53.136452 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefadf7-44ba-4a4f-a68c-df5f280be68d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:53.136752 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefadf7-44ba-4a4f-a68c-df5f280be68d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:53.147646 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fefadf7-44ba-4a4f-a68c-df5f280be68d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:45:53.152579 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-7fefadf7-44ba-4a4f-a68c-df5f280be68d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 10:45:53.152579 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:53.152579 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:53.152579 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:53.152579 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:53.152579 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:53.152579 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:53.152579 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:53.152579 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:53.152579 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:53.152579 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:53.152579 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:53.152579 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:45:53.152579 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:45:53.152579 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:45:53.152579 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:45:53.153477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:45:53.153477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:45:53.153477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:45:53.153477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:45:53.153477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:53.153477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:53.153477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:53.153477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:53.153477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:53.153477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:53.153477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:53.153477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:53.153477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:53.153477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:53.153477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:53.153477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:53.154377 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:53.154377 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:53.154377 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:53.154377 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:53.154377 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:45:53.154377 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:45:53.154377 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:45:53.154377 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:45:53.154377 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:45:53.154377 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:45:53.154377 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 10:45:53.154377 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:53.154377 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 516/1029] 10.222.0.16 () {56 vars in 1120 bytes} [Wed Aug 31 10:45:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:53.160378 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba06626-2df4-447c-b9a6-6f070fa6bd2f None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:53.169493 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fefadf7-44ba-4a4f-a68c-df5f280be68d None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:53.258997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:53.276661 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba06626-2df4-447c-b9a6-6f070fa6bd2f None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:53.281091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fefadf7-44ba-4a4f-a68c-df5f280be68d None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:53.288565 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:53.318188 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7042c8b-928b-485d-89d3-8c803e05c960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:53.319150 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7042c8b-928b-485d-89d3-8c803e05c960 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:53.319419 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7042c8b-928b-485d-89d3-8c803e05c960 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:53.319693 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7042c8b-928b-485d-89d3-8c803e05c960 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:53.320365 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7042c8b-928b-485d-89d3-8c803e05c960 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:53.320670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7042c8b-928b-485d-89d3-8c803e05c960 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:53.322042 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7042c8b-928b-485d-89d3-8c803e05c960 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:53.322158 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ca444-6d71-4efb-a50b-8486c1e2ccde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:53.323287 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ca444-6d71-4efb-a50b-8486c1e2ccde None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:53.323639 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ca444-6d71-4efb-a50b-8486c1e2ccde None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:53.323939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ca444-6d71-4efb-a50b-8486c1e2ccde None admin] PATH_INFO: `/v3/services/image` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:53.324598 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95ca444-6d71-4efb-a50b-8486c1e2ccde None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:53.326065 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95ca444-6d71-4efb-a50b-8486c1e2ccde None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:53.336522 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 514/1030] 10.222.0.16 () {56 vars in 1115 bytes} [Wed Aug 31 10:45:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:53.336674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-c95ca444-6d71-4efb-a50b-8486c1e2ccde None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 10:45:53.336674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:53.336674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:45:53.336674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:45:53.336674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:45:53.336674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:45:53.336674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:45:53.336674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:53.336674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 10:45:53.336674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:53.336674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:45:53.336674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:53.336674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:53.336674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:53.336674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:53.336674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:53.336674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:53.337595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:53.337595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:53.337595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:53.337595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:53.337595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:53.337595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:53.337595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:45:53.337595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:45:53.337595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:45:53.337595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:45:53.337595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:53.337595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:53.337595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:53.337595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:53.337595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:53.337595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:53.339283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:53.339283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:53.339283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:53.339283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:53.339283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:53.339283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:53.339283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:53.339283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:53.339283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:53.339283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:53.339283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:45:53.339283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:53.339283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 10:45:53.339283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:53.339283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 517/1031] 10.222.0.16 () {56 vars in 1105 bytes} [Wed Aug 31 10:45:53 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:53.342976 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7042c8b-928b-485d-89d3-8c803e05c960 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:53.343549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ca444-6d71-4efb-a50b-8486c1e2ccde None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:53.449982 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7042c8b-928b-485d-89d3-8c803e05c960 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:53.453395 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ca444-6d71-4efb-a50b-8486c1e2ccde None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:53.486260 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400181b-6752-4940-9bff-0d8a2c9b4c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:53.486925 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400181b-6752-4940-9bff-0d8a2c9b4c50 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:53.487205 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400181b-6752-4940-9bff-0d8a2c9b4c50 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:53.487480 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400181b-6752-4940-9bff-0d8a2c9b4c50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:53.489977 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea117544-3d3b-4730-8582-0a1c0c4dcef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:53.490615 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea117544-3d3b-4730-8582-0a1c0c4dcef1 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:53.490897 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea117544-3d3b-4730-8582-0a1c0c4dcef1 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:53.491173 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea117544-3d3b-4730-8582-0a1c0c4dcef1 None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:53.491807 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea117544-3d3b-4730-8582-0a1c0c4dcef1 None admin] RBAC: Adding query filter params (name=image) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:53.492112 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea117544-3d3b-4730-8582-0a1c0c4dcef1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:53.493492 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea117544-3d3b-4730-8582-0a1c0c4dcef1 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:53.499883 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6400181b-6752-4940-9bff-0d8a2c9b4c50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:53.500051 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 518/1032] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:53.501343 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6400181b-6752-4940-9bff-0d8a2c9b4c50 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:53.505589 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea117544-3d3b-4730-8582-0a1c0c4dcef1 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:53.513249 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:53.520231 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-6400181b-6752-4940-9bff-0d8a2c9b4c50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:53.520231 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:53.520231 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:53.520231 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:53.520231 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:53.520231 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:53.520231 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:53.520231 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:53.520231 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:53.520231 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:53.520231 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:53.520231 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:53.520231 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:53.520231 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:53.520231 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:53.520231 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:53.521310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:53.521310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:53.521310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:53.521310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:53.521310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:53.521310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:53.521310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:53.521310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:53.521310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:53.521310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:53.521310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:53.521310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:53.521310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:53.521310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:53.521310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:53.521310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:53.521310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:53.523043 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:53.523043 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:53.523043 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:53.523043 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 515/1033] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:53.528214 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400181b-6752-4940-9bff-0d8a2c9b4c50 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:53.546067 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:53.619381 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea117544-3d3b-4730-8582-0a1c0c4dcef1 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:53.639396 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400181b-6752-4940-9bff-0d8a2c9b4c50 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:53.655302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c566141b-a618-4aa1-9aed-d795cb89a6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:53.655915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566141b-a618-4aa1-9aed-d795cb89a6cf None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:53.656299 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566141b-a618-4aa1-9aed-d795cb89a6cf None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:53.656573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566141b-a618-4aa1-9aed-d795cb89a6cf None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:53.657397 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c566141b-a618-4aa1-9aed-d795cb89a6cf None admin] RBAC: Adding query filter params (type=image) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:53.657665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c566141b-a618-4aa1-9aed-d795cb89a6cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:53.659111 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c566141b-a618-4aa1-9aed-d795cb89a6cf None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:53.670938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 519/1034] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:53 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:53.672482 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c566141b-a618-4aa1-9aed-d795cb89a6cf None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:53.680483 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde33fe4-24ba-45d3-8168-648966168d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:53.681111 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde33fe4-24ba-45d3-8168-648966168d71 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:53.681406 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde33fe4-24ba-45d3-8168-648966168d71 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:53.681681 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde33fe4-24ba-45d3-8168-648966168d71 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:53.682552 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde33fe4-24ba-45d3-8168-648966168d71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:53.682718 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde33fe4-24ba-45d3-8168-648966168d71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:53.684078 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde33fe4-24ba-45d3-8168-648966168d71 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:53.698430 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 516/1035] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:53.704024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde33fe4-24ba-45d3-8168-648966168d71 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:53.773592 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:53.779456 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:53.783587 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c566141b-a618-4aa1-9aed-d795cb89a6cf None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:53.820575 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde33fe4-24ba-45d3-8168-648966168d71 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:53.824464 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf56a93-2a2d-4f27-a9a0-a861a9b91a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:53.825073 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf56a93-2a2d-4f27-a9a0-a861a9b91a6d None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:53.825337 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf56a93-2a2d-4f27-a9a0-a861a9b91a6d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:53.825606 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf56a93-2a2d-4f27-a9a0-a861a9b91a6d None admin] PATH_INFO: `/v3/endpoints` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:53.826297 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf56a93-2a2d-4f27-a9a0-a861a9b91a6d None admin] RBAC: Adding query filter params (interface=public, service_id=3d355d2313ae462784168dee00d75fe2, region_id=RegionOne) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:53.826625 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf56a93-2a2d-4f27-a9a0-a861a9b91a6d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:53.827985 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf56a93-2a2d-4f27-a9a0-a861a9b91a6d None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:53.833066 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 520/1036] 10.222.0.16 () {56 vars in 1251 bytes} [Wed Aug 31 10:45:53 2022] GET /identity/v3/endpoints?service_id=3d355d2313ae462784168dee00d75fe2&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:53.839335 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf56a93-2a2d-4f27-a9a0-a861a9b91a6d None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:53.871075 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b6a6b-d7b6-409c-b062-0dbc0dae0fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:53.872263 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b6a6b-d7b6-409c-b062-0dbc0dae0fe1 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:53.872732 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b6a6b-d7b6-409c-b062-0dbc0dae0fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:53.873147 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b6a6b-d7b6-409c-b062-0dbc0dae0fe1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:53.882371 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b6a6b-d7b6-409c-b062-0dbc0dae0fe1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:53.884631 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b6a6b-d7b6-409c-b062-0dbc0dae0fe1 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:53.899040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-ef7b6a6b-d7b6-409c-b062-0dbc0dae0fe1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:45:53.899040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:53.899040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:53.899040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:53.899040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:53.899040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:53.899040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:53.899040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:53.899040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:53.899040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:53.899040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:53.899040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:53.899040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:45:53.899040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:45:53.899040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:45:53.899040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:45:53.899983 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:53.899983 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:53.899983 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:45:53.899983 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:53.899983 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:45:53.899983 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:53.899983 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:53.899983 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:53.899983 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:53.899983 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:53.899983 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:53.899983 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:53.899983 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:53.899983 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:53.899983 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:53.899983 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:53.899983 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:53.899983 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:53.900996 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:53.900996 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:53.900996 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:45:53.900996 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:45:53.900996 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:45:53.900996 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:45:53.900996 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:45:53.900996 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:45:53.900996 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:45:53.900996 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:53.900996 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 517/1037] 10.222.0.16 () {56 vars in 1093 bytes} [Wed Aug 31 10:45:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:53.906764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b6a6b-d7b6-409c-b062-0dbc0dae0fe1 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:53.944593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf56a93-2a2d-4f27-a9a0-a861a9b91a6d None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:53.981971 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4da36f-3953-4a8c-921b-35e325f9e73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:53.982550 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4da36f-3953-4a8c-921b-35e325f9e73b None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:53.982856 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4da36f-3953-4a8c-921b-35e325f9e73b None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:53.983077 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4da36f-3953-4a8c-921b-35e325f9e73b None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:53.984226 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4da36f-3953-4a8c-921b-35e325f9e73b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:53.985045 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4da36f-3953-4a8c-921b-35e325f9e73b None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:53.991986 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 521/1038] 10.222.0.16 () {56 vars in 1087 bytes} [Wed Aug 31 10:45:53 2022] GET /identity/v3/services => generated 1957 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 10:45:54.018714 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b6a6b-d7b6-409c-b062-0dbc0dae0fe1 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:54.043443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:54.058583 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2cf7b8-f636-4df4-be79-cfd9e19db4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:54.059216 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2cf7b8-f636-4df4-be79-cfd9e19db4b5 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:54.059492 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2cf7b8-f636-4df4-be79-cfd9e19db4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:54.059763 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2cf7b8-f636-4df4-be79-cfd9e19db4b5 None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:54.060515 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2cf7b8-f636-4df4-be79-cfd9e19db4b5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:54.060819 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2cf7b8-f636-4df4-be79-cfd9e19db4b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:54.062263 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2cf7b8-f636-4df4-be79-cfd9e19db4b5 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:54.086423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 518/1039] 10.222.0.16 () {56 vars in 1133 bytes} [Wed Aug 31 10:45:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:54.090139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4da36f-3953-4a8c-921b-35e325f9e73b None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:54.125708 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:54.202347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4da36f-3953-4a8c-921b-35e325f9e73b None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:54.238579 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241a0d7-08c0-4552-a656-ebf1fbafe071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:54.239531 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241a0d7-08c0-4552-a656-ebf1fbafe071 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:54.240006 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241a0d7-08c0-4552-a656-ebf1fbafe071 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:54.240468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241a0d7-08c0-4552-a656-ebf1fbafe071 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:54.313900 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4241a0d7-08c0-4552-a656-ebf1fbafe071 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:54.315414 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4241a0d7-08c0-4552-a656-ebf1fbafe071 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:54.342605 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:54.416757 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 522/1040] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:45:54 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:45:54.422594 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2cf7b8-f636-4df4-be79-cfd9e19db4b5 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:54.529901 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3cd1b-a283-473f-b2c2-dd2df5bff403 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:54.530084 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2cf7b8-f636-4df4-be79-cfd9e19db4b5 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:54.530183 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3cd1b-a283-473f-b2c2-dd2df5bff403 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:54.530443 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3cd1b-a283-473f-b2c2-dd2df5bff403 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:54.532186 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 523/1041] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:45:54.536951 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf59c1-adda-4f58-bb4d-cafdad0d2542 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:54.537232 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf59c1-adda-4f58-bb4d-cafdad0d2542 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:54.537500 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf59c1-adda-4f58-bb4d-cafdad0d2542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:54.555877 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-e3bf59c1-adda-4f58-bb4d-cafdad0d2542 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:54.557742 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-e3bf59c1-adda-4f58-bb4d-cafdad0d2542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:54.568499 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaaf479-73d6-45f3-b920-98d5aee964e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:54.569129 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaaf479-73d6-45f3-b920-98d5aee964e4 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:54.569403 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaaf479-73d6-45f3-b920-98d5aee964e4 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:54.569668 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaaf479-73d6-45f3-b920-98d5aee964e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:54.580458 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaaf479-73d6-45f3-b920-98d5aee964e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:54.581892 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaaf479-73d6-45f3-b920-98d5aee964e4 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:54.598192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-0aaaf479-73d6-45f3-b920-98d5aee964e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:54.598192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:54.598192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:54.598192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:54.598192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:54.598192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:54.598192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:54.598192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:54.598192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:54.598192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:54.598192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:54.598192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:54.598192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:54.598192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:54.598192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:54.598192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:54.599198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:54.599198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:54.599198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:54.599198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:54.599198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:54.599198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:54.599198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:54.599198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:54.599198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:54.599198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:54.599198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:54.599198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:54.599198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:54.599198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:54.599198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:54.599198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:54.599198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:54.600464 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:54.600464 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:54.600464 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:54.600464 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 519/1042] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:54.605611 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaaf479-73d6-45f3-b920-98d5aee964e4 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:54.635899 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 524/1043] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:45:54.639062 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:54.710247 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaaf479-73d6-45f3-b920-98d5aee964e4 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:54.745019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e01cc-5c9e-4cfe-85e2-b5c779405bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:54.745911 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e01cc-5c9e-4cfe-85e2-b5c779405bd7 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:54.746341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e01cc-5c9e-4cfe-85e2-b5c779405bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:54.746819 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e01cc-5c9e-4cfe-85e2-b5c779405bd7 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:54.747604 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e01cc-5c9e-4cfe-85e2-b5c779405bd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:54.748068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e01cc-5c9e-4cfe-85e2-b5c779405bd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:54.749685 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e01cc-5c9e-4cfe-85e2-b5c779405bd7 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:54.761965 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 520/1044] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:54.767518 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf59c1-adda-4f58-bb4d-cafdad0d2542 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:54.824000 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:54.868062 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf59c1-adda-4f58-bb4d-cafdad0d2542 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:54.903766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783fc0d-6c32-489b-bf0d-e71047575613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:54.904406 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783fc0d-6c32-489b-bf0d-e71047575613 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:54.904672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783fc0d-6c32-489b-bf0d-e71047575613 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:54.904945 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783fc0d-6c32-489b-bf0d-e71047575613 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:54.907808 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e01cc-5c9e-4cfe-85e2-b5c779405bd7 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:54.913127 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783fc0d-6c32-489b-bf0d-e71047575613 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:54.914612 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783fc0d-6c32-489b-bf0d-e71047575613 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:54.926333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-2783fc0d-6c32-489b-bf0d-e71047575613 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:54.926333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:54.926333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:54.926333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:54.926333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:54.926333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:54.926333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:54.926333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:54.926333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:54.926333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:54.926333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:54.926333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:54.926333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:45:54.926333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:45:54.926333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:45:54.927359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:45:54.927359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:54.927359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:54.927359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:54.927359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:54.927359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:54.927359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:54.927359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:54.927359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:54.927359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:54.927359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:54.927359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:54.927359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:54.927359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:54.927359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:54.927359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:54.927359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:54.928245 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:45:54.928245 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:45:54.928245 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:45:54.928245 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:45:54.928245 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:45:54.928245 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:45:54.928245 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:54.928245 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:54.928245 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 525/1045] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:45:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:54.933649 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783fc0d-6c32-489b-bf0d-e71047575613 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:55.004919 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:55.015739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e01cc-5c9e-4cfe-85e2-b5c779405bd7 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:55.045649 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783fc0d-6c32-489b-bf0d-e71047575613 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:55.053076 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5ba79-544f-4c02-8674-620f362917f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:55.054116 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5ba79-544f-4c02-8674-620f362917f1 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:55.054409 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5ba79-544f-4c02-8674-620f362917f1 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:55.054690 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5ba79-544f-4c02-8674-620f362917f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:55.063094 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:55.066825 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a5ba79-544f-4c02-8674-620f362917f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:45:55.071620 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-e8a5ba79-544f-4c02-8674-620f362917f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:45:55.071620 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:55.071620 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:55.071620 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:55.071620 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:55.071620 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:55.071620 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:55.071620 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:55.071620 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:55.071620 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:55.071620 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:55.071620 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:55.071620 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:45:55.071620 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:45:55.071620 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:45:55.071620 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:45:55.072943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:45:55.072943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:45:55.072943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:45:55.072943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:45:55.072943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:55.072943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:55.072943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:55.072943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:55.072943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:55.072943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:55.072943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:55.072943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:55.072943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:55.072943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:55.072943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:55.072943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:55.073837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:55.073837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:55.073837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:55.073837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:55.073837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:45:55.073837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:45:55.073837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:45:55.073837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:45:55.073837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:45:55.073837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:45:55.073837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:45:55.073837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:55.073837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 521/1046] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:45:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:45:55.079019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5ba79-544f-4c02-8674-620f362917f1 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:55.083997 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9e046-2e8b-4cc6-b2b2-734f9a47e6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:55.084681 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9e046-2e8b-4cc6-b2b2-734f9a47e6d1 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:55.084955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9e046-2e8b-4cc6-b2b2-734f9a47e6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:55.085223 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9e046-2e8b-4cc6-b2b2-734f9a47e6d1 None admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:55.085911 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9e046-2e8b-4cc6-b2b2-734f9a47e6d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:55.086211 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9e046-2e8b-4cc6-b2b2-734f9a47e6d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:55.087673 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9e046-2e8b-4cc6-b2b2-734f9a47e6d1 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:55.105949 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 526/1047] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:45:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:55.112508 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9e046-2e8b-4cc6-b2b2-734f9a47e6d1 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:55.197222 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5ba79-544f-4c02-8674-620f362917f1 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:55.227335 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9e046-2e8b-4cc6-b2b2-734f9a47e6d1 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:55.233790 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c50b33d-34e3-4801-817d-8e52b253ee42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:55.234409 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50b33d-34e3-4801-817d-8e52b253ee42 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:55.234698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50b33d-34e3-4801-817d-8e52b253ee42 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:55.234963 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50b33d-34e3-4801-817d-8e52b253ee42 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:55.235620 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c50b33d-34e3-4801-817d-8e52b253ee42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:55.235922 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c50b33d-34e3-4801-817d-8e52b253ee42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:55.237273 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c50b33d-34e3-4801-817d-8e52b253ee42 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:55.250550 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 522/1048] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:45:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:55.252969 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c50b33d-34e3-4801-817d-8e52b253ee42 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:55.265295 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56a8f8-3b5a-484e-b186-47dcc08182d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:55.271992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56a8f8-3b5a-484e-b186-47dcc08182d3 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:55.271992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56a8f8-3b5a-484e-b186-47dcc08182d3 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:55.271992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56a8f8-3b5a-484e-b186-47dcc08182d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:55.271992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c56a8f8-3b5a-484e-b186-47dcc08182d3 None admin] RBAC: Adding query filter params (role.id=1715f4cc50354636b1f7c3a9864e1e80, scope.project.id=8db9b3f8a9f041938ac7b39003a35015, user.id=f11971a594014d86968c3356d11d42f7) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:55.271992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c56a8f8-3b5a-484e-b186-47dcc08182d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:55.271992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c56a8f8-3b5a-484e-b186-47dcc08182d3 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:55.272473 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:55.274424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 527/1049] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:45:55 2022] GET /identity/v3/role_assignments?user.id=f11971a594014d86968c3356d11d42f7&scope.project.id=8db9b3f8a9f041938ac7b39003a35015&role.id=1715f4cc50354636b1f7c3a9864e1e80 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:55.371438 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c50b33d-34e3-4801-817d-8e52b253ee42 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:55.409757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d8f9c-385e-4fa5-8f86-a78fdc5ea7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:55.410424 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d8f9c-385e-4fa5-8f86-a78fdc5ea7f1 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:55.410705 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d8f9c-385e-4fa5-8f86-a78fdc5ea7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:55.410976 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d8f9c-385e-4fa5-8f86-a78fdc5ea7f1 None admin] PATH_INFO: `/v3/users/94f1fc3eb1ab4fe3a9386f6121033cca` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:55.427311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d8f9c-385e-4fa5-8f86-a78fdc5ea7f1 None admin] RBAC: Authorizing `identity:get_user(user_id=94f1fc3eb1ab4fe3a9386f6121033cca)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:55.428625 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d8f9c-385e-4fa5-8f86-a78fdc5ea7f1 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:55.430505 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 523/1050] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:45:55 2022] GET /identity/v3/users/94f1fc3eb1ab4fe3a9386f6121033cca => generated 285 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:55.436823 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56a8f8-3b5a-484e-b186-47dcc08182d3 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:55.481960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:55.544838 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56a8f8-3b5a-484e-b186-47dcc08182d3 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:55.581532 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61114a6-2021-4cc4-8e8d-d892e86c9caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:55.581532 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61114a6-2021-4cc4-8e8d-d892e86c9caa None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:55.583286 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61114a6-2021-4cc4-8e8d-d892e86c9caa None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:55.583286 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61114a6-2021-4cc4-8e8d-d892e86c9caa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:55.657519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61114a6-2021-4cc4-8e8d-d892e86c9caa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:55.658930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61114a6-2021-4cc4-8e8d-d892e86c9caa None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:55.703123 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:55.771801 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 528/1051] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:45:55 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:45:55.779137 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d8f9c-385e-4fa5-8f86-a78fdc5ea7f1 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:55.853649 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:55.887045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d8f9c-385e-4fa5-8f86-a78fdc5ea7f1 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:55.922793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95e46b-be2e-4436-a34c-e0bcb320542e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:55.923433 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95e46b-be2e-4436-a34c-e0bcb320542e None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:55.923657 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95e46b-be2e-4436-a34c-e0bcb320542e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:55.923878 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95e46b-be2e-4436-a34c-e0bcb320542e None admin] PATH_INFO: `/v3/projects/85b7a190e444441c8361d1f925e0dcff` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:55.925218 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95e46b-be2e-4436-a34c-e0bcb320542e None admin] RBAC: Authorizing `identity:get_project(project_id=85b7a190e444441c8361d1f925e0dcff)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:55.926769 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95e46b-be2e-4436-a34c-e0bcb320542e None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:55.933979 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 524/1052] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:45:55 2022] GET /identity/v3/projects/85b7a190e444441c8361d1f925e0dcff => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:55.935313 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61114a6-2021-4cc4-8e8d-d892e86c9caa None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:55.945121 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ba96d-263f-443f-a930-1fdea6aa8442 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:55.945397 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ba96d-263f-443f-a930-1fdea6aa8442 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:55.945674 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ba96d-263f-443f-a930-1fdea6aa8442 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:55.947341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 525/1053] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:45:55.952789 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b3c40-8465-405d-b8ec-c97d5b7c17b2 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:55.953236 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b3c40-8465-405d-b8ec-c97d5b7c17b2 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:55.953509 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b3c40-8465-405d-b8ec-c97d5b7c17b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:55.975644 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-844b3c40-8465-405d-b8ec-c97d5b7c17b2 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:55.975644 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-844b3c40-8465-405d-b8ec-c97d5b7c17b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:56.048962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61114a6-2021-4cc4-8e8d-d892e86c9caa None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:56.050532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 526/1054] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:45:56.088615 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8779f9-664a-4718-aa17-d069db3e997d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:56.089214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8779f9-664a-4718-aa17-d069db3e997d None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:56.089487 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8779f9-664a-4718-aa17-d069db3e997d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:56.089783 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8779f9-664a-4718-aa17-d069db3e997d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:56.090495 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8779f9-664a-4718-aa17-d069db3e997d None admin] RBAC: Adding query filter params (role.id=13ca33d694c645d6920930d39dc52a0d, scope.project.id=85b7a190e444441c8361d1f925e0dcff, user.id=94f1fc3eb1ab4fe3a9386f6121033cca) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:56.090799 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8779f9-664a-4718-aa17-d069db3e997d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:56.092209 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8779f9-664a-4718-aa17-d069db3e997d None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:56.101892 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 529/1055] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:45:55 2022] GET /identity/v3/role_assignments?user.id=94f1fc3eb1ab4fe3a9386f6121033cca&scope.project.id=85b7a190e444441c8361d1f925e0dcff&role.id=13ca33d694c645d6920930d39dc52a0d => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:56.326342 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b3c40-8465-405d-b8ec-c97d5b7c17b2 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:56.403934 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:56.438660 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b3c40-8465-405d-b8ec-c97d5b7c17b2 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:56.475004 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060129cc-6f80-49cf-ae80-5eb4df469581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:56.475636 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060129cc-6f80-49cf-ae80-5eb4df469581 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:56.475909 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060129cc-6f80-49cf-ae80-5eb4df469581 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:56.476174 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060129cc-6f80-49cf-ae80-5eb4df469581 None admin] PATH_INFO: `/v3/services/image` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:56.476753 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060129cc-6f80-49cf-ae80-5eb4df469581 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:56.478103 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060129cc-6f80-49cf-ae80-5eb4df469581 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:56.487019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-060129cc-6f80-49cf-ae80-5eb4df469581 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 10:45:56.487019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:56.487019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:45:56.487019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:45:56.487019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:45:56.487019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:45:56.487019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:45:56.487019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:56.487019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 10:45:56.487019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:56.487019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:45:56.487019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:56.487019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:56.487019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:56.487019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:56.487019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:56.487019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:56.487987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:56.487987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:56.487987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:56.487987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:56.487987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:56.487987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:56.487987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:45:56.487987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:45:56.487987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:45:56.487987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:45:56.487987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:56.487987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:56.487987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:56.487987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:56.487987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:56.487987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:56.488823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:56.488823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:56.488823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:56.488823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:56.488823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:56.488823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:56.488823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:56.488823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:56.488823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:56.488823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:56.488823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:45:56.488823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:45:56.488823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 10:45:56.488823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:56.493159 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 527/1056] 10.222.0.16 () {56 vars in 1105 bytes} [Wed Aug 31 10:45:56 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:56.494379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8779f9-664a-4718-aa17-d069db3e997d None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:56.499599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:56.599104 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8779f9-664a-4718-aa17-d069db3e997d None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:56.632778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf6c6d-9c67-42cb-a975-e8e797a574fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:56.633383 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf6c6d-9c67-42cb-a975-e8e797a574fe None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:56.633656 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf6c6d-9c67-42cb-a975-e8e797a574fe None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:56.633922 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf6c6d-9c67-42cb-a975-e8e797a574fe None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:56.634573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdf6c6d-9c67-42cb-a975-e8e797a574fe None admin] RBAC: Adding query filter params (name=image) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:56.634872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdf6c6d-9c67-42cb-a975-e8e797a574fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:56.636266 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdf6c6d-9c67-42cb-a975-e8e797a574fe None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:56.641452 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 530/1057] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:56.646971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060129cc-6f80-49cf-ae80-5eb4df469581 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:56.759961 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060129cc-6f80-49cf-ae80-5eb4df469581 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:56.791303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:56.797962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370706b7-0c60-4245-8637-66afd1340f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:56.803987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370706b7-0c60-4245-8637-66afd1340f33 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:56.803987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370706b7-0c60-4245-8637-66afd1340f33 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:56.803987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370706b7-0c60-4245-8637-66afd1340f33 None admin] PATH_INFO: `/v3/services` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:56.803987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370706b7-0c60-4245-8637-66afd1340f33 None admin] RBAC: Adding query filter params (type=image) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:56.803987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370706b7-0c60-4245-8637-66afd1340f33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:56.803987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370706b7-0c60-4245-8637-66afd1340f33 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:56.811408 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 528/1058] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:56.812723 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf6c6d-9c67-42cb-a975-e8e797a574fe None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:56.882673 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:56.917604 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf6c6d-9c67-42cb-a975-e8e797a574fe None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:56.953971 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48d3ed-a1cd-4462-b4b7-e3a72f00df22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:56.955451 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48d3ed-a1cd-4462-b4b7-e3a72f00df22 None admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:56.955954 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48d3ed-a1cd-4462-b4b7-e3a72f00df22 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:56.956427 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48d3ed-a1cd-4462-b4b7-e3a72f00df22 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:56.957379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb48d3ed-a1cd-4462-b4b7-e3a72f00df22 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:56.959023 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb48d3ed-a1cd-4462-b4b7-e3a72f00df22 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:57.052536 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 531/1059] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 10:45:56 2022] POST /identity/v3/endpoints => generated 327 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:45:57.247344 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b335b6-37b8-430f-ba85-a41828355c8a None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:57.247894 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b335b6-37b8-430f-ba85-a41828355c8a None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:57.247894 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b335b6-37b8-430f-ba85-a41828355c8a None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:57.249245 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 529/1060] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:57.254946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db6622-ad3e-4904-adde-ab7b0e415824 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:57.255232 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db6622-ad3e-4904-adde-ab7b0e415824 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:57.255508 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db6622-ad3e-4904-adde-ab7b0e415824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:57.275397 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-69db6622-ad3e-4904-adde-ab7b0e415824 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:57.276638 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-69db6622-ad3e-4904-adde-ab7b0e415824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:57.355101 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 532/1061] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:45:57.625739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b335b6-37b8-430f-ba85-a41828355c8a None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:57.734902 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b335b6-37b8-430f-ba85-a41828355c8a None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:57.769728 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9df00-2332-4626-9766-663533c220d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:57.770353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9df00-2332-4626-9766-663533c220d8 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:57.770654 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9df00-2332-4626-9766-663533c220d8 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:57.770922 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9df00-2332-4626-9766-663533c220d8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:57.781323 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9df00-2332-4626-9766-663533c220d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:45:57.786052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-2fd9df00-2332-4626-9766-663533c220d8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 10:45:57.786052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:57.786052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:57.786052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:57.786052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:57.786052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:57.786052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:57.786052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:57.786052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:57.786052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:57.786052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:57.786052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:57.786052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:45:57.786052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:45:57.786052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:45:57.786052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:45:57.787211 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:45:57.787211 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:45:57.787211 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:45:57.787211 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:45:57.787211 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:57.787211 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:57.787211 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:57.787211 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:57.787211 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:57.787211 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:57.787211 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:57.787211 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:57.787211 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:57.787211 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:57.787211 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:57.787211 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:57.788150 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:57.788150 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:57.788150 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:57.788150 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:57.788150 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:45:57.788150 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:45:57.788150 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:45:57.788150 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:45:57.788150 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:45:57.788150 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:45:57.788150 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 10:45:57.788150 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:57.790726 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 530/1062] 10.222.0.16 () {56 vars in 1120 bytes} [Wed Aug 31 10:45:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:57.794424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db6622-ad3e-4904-adde-ab7b0e415824 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:57.847606 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:57.900401 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db6622-ad3e-4904-adde-ab7b0e415824 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:57.942331 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c7cbcf-24d5-442a-8700-ea15ee65b043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:57.942972 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7cbcf-24d5-442a-8700-ea15ee65b043 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:57.943246 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7cbcf-24d5-442a-8700-ea15ee65b043 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:57.943514 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7cbcf-24d5-442a-8700-ea15ee65b043 None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:57.944175 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c7cbcf-24d5-442a-8700-ea15ee65b043 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:57.944473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c7cbcf-24d5-442a-8700-ea15ee65b043 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:57.945830 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c7cbcf-24d5-442a-8700-ea15ee65b043 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:57.960028 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 533/1063] 10.222.0.16 () {56 vars in 1115 bytes} [Wed Aug 31 10:45:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:57.961721 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9df00-2332-4626-9766-663533c220d8 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:58.033996 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:58.040689 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202fb87-1b2c-440b-a3f8-3f4e4a51a052 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:58.041480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202fb87-1b2c-440b-a3f8-3f4e4a51a052 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:58.047080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202fb87-1b2c-440b-a3f8-3f4e4a51a052 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:58.047080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 534/1064] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:45:58.049836 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df236700-7fb6-42bc-a91e-d9921d796221 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:58.050110 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df236700-7fb6-42bc-a91e-d9921d796221 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:58.050378 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df236700-7fb6-42bc-a91e-d9921d796221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:58.069434 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-df236700-7fb6-42bc-a91e-d9921d796221 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:58.070702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-df236700-7fb6-42bc-a91e-d9921d796221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:58.084650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9df00-2332-4626-9766-663533c220d8 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:58.126857 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c5071-7b78-4f3d-bc2b-661d16a013cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:58.127505 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c5071-7b78-4f3d-bc2b-661d16a013cc None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:58.127783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c5071-7b78-4f3d-bc2b-661d16a013cc None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:58.128052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c5071-7b78-4f3d-bc2b-661d16a013cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:58.139515 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c5071-7b78-4f3d-bc2b-661d16a013cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:58.140979 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c5071-7b78-4f3d-bc2b-661d16a013cc None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:58.158506 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 535/1065] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:45:58.161813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-eb8c5071-7b78-4f3d-bc2b-661d16a013cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:58.161813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:58.161813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:58.161813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:58.161813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:58.161813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:58.161813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:58.161813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:58.161813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:58.161813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:58.161813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:58.161813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:58.161813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:58.161813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:58.161813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:58.161813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:58.162752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:58.162752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:58.162752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:58.162752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:58.162752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:58.162752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:58.162752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:58.162752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:58.162752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:58.162752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:58.162752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:58.162752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:58.162752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:58.162752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:58.162752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:58.162752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:58.162752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:58.163998 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:58.163998 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:58.163998 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:58.163998 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 531/1066] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:58.172934 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df236700-7fb6-42bc-a91e-d9921d796221 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:58.219933 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:58.277517 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df236700-7fb6-42bc-a91e-d9921d796221 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:58.312526 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd177bf3-1bd5-4f57-aa12-91337fa932cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:58.313137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd177bf3-1bd5-4f57-aa12-91337fa932cd None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:58.313414 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd177bf3-1bd5-4f57-aa12-91337fa932cd None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:58.313682 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd177bf3-1bd5-4f57-aa12-91337fa932cd None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:58.314325 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd177bf3-1bd5-4f57-aa12-91337fa932cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:58.314651 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd177bf3-1bd5-4f57-aa12-91337fa932cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:58.316018 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd177bf3-1bd5-4f57-aa12-91337fa932cd None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:58.332525 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 536/1067] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:58.334096 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c5071-7b78-4f3d-bc2b-661d16a013cc None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:58.433496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd177bf3-1bd5-4f57-aa12-91337fa932cd None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:58.448341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c5071-7b78-4f3d-bc2b-661d16a013cc None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:58.457898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:58.485298 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3987298-94de-4fac-badb-f415647fa2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:58.486208 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3987298-94de-4fac-badb-f415647fa2ec None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:58.486688 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3987298-94de-4fac-badb-f415647fa2ec None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:58.486965 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3987298-94de-4fac-badb-f415647fa2ec None admin] PATH_INFO: `/v3/users/nova` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:58.497842 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3987298-94de-4fac-badb-f415647fa2ec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:58.499867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3987298-94de-4fac-badb-f415647fa2ec None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:58.514192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-a3987298-94de-4fac-badb-f415647fa2ec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:45:58.514192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:58.514192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:58.514192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:58.514192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:58.514192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:58.514192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:58.514192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:58.514192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:58.514192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:58.514192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:58.514192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:58.514192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:45:58.514192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:45:58.514192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:45:58.514192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:45:58.515217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:58.515217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:58.515217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:45:58.515217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:58.515217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:45:58.515217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:45:58.515217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:58.515217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:58.515217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:58.515217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:58.515217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:58.515217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:58.515217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:58.515217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:58.515217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:58.515217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:58.515217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:58.515217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:58.516197 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:58.516197 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:58.516197 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:45:58.516197 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:45:58.516197 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:45:58.516197 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:45:58.516197 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:45:58.516197 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:45:58.516197 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:45:58.516197 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:58.516981 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 532/1068] 10.222.0.16 () {56 vars in 1093 bytes} [Wed Aug 31 10:45:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:58.521296 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3987298-94de-4fac-badb-f415647fa2ec None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:58.581403 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd177bf3-1bd5-4f57-aa12-91337fa932cd None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:58.583495 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:58.626867 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156930f4-a869-4778-b94c-a50020a7a79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:58.627503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156930f4-a869-4778-b94c-a50020a7a79a None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:58.627845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156930f4-a869-4778-b94c-a50020a7a79a None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:58.628121 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156930f4-a869-4778-b94c-a50020a7a79a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:58.641272 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156930f4-a869-4778-b94c-a50020a7a79a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:45:58.643676 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3987298-94de-4fac-badb-f415647fa2ec None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:58.646175 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-156930f4-a869-4778-b94c-a50020a7a79a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:45:58.646175 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:58.646175 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:58.646175 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:58.646175 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:58.646175 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:58.646175 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:58.646175 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:58.646175 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:58.646175 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:58.646175 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:58.646175 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:58.646175 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:45:58.646175 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:45:58.646175 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:45:58.646175 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:45:58.647141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:45:58.647141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:45:58.647141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:45:58.647141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:45:58.647141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:58.647141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:58.647141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:58.647141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:58.647141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:58.647141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:58.647141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:58.647141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:58.647141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:58.647141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:58.647141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:58.647141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:58.647976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:58.647976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:58.647976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:58.647976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:58.647976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:45:58.647976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:45:58.647976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:45:58.647976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:45:58.647976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:45:58.647976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:45:58.647976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:45:58.647976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:45:58.647976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 537/1069] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:45:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:58.653630 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156930f4-a869-4778-b94c-a50020a7a79a None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:58.685212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614024e1-2ce6-479a-b943-9d45a8627b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:58.685757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614024e1-2ce6-479a-b943-9d45a8627b4b None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:58.686029 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614024e1-2ce6-479a-b943-9d45a8627b4b None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:58.686297 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614024e1-2ce6-479a-b943-9d45a8627b4b None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:58.687076 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614024e1-2ce6-479a-b943-9d45a8627b4b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:58.687379 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614024e1-2ce6-479a-b943-9d45a8627b4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:58.688840 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614024e1-2ce6-479a-b943-9d45a8627b4b None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:58.711831 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 533/1070] 10.222.0.16 () {56 vars in 1133 bytes} [Wed Aug 31 10:45:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:58.712982 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614024e1-2ce6-479a-b943-9d45a8627b4b None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:58.744164 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:58.774912 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156930f4-a869-4778-b94c-a50020a7a79a None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:58.815168 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968909e-7c61-4bcc-8bdb-604a34670a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:58.815781 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968909e-7c61-4bcc-8bdb-604a34670a32 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:58.816060 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968909e-7c61-4bcc-8bdb-604a34670a32 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:58.816332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968909e-7c61-4bcc-8bdb-604a34670a32 None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:58.817004 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968909e-7c61-4bcc-8bdb-604a34670a32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:58.817307 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968909e-7c61-4bcc-8bdb-604a34670a32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:58.818706 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968909e-7c61-4bcc-8bdb-604a34670a32 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:58.826473 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614024e1-2ce6-479a-b943-9d45a8627b4b None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:58.830439 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 538/1071] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:45:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:58.833398 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968909e-7c61-4bcc-8bdb-604a34670a32 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:58.862428 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:58.871674 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877721ad-fc33-4725-b506-1e019045d450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:58.872281 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877721ad-fc33-4725-b506-1e019045d450 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:58.872564 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877721ad-fc33-4725-b506-1e019045d450 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:58.872815 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877721ad-fc33-4725-b506-1e019045d450 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:58.884411 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877721ad-fc33-4725-b506-1e019045d450 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:58.886872 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877721ad-fc33-4725-b506-1e019045d450 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:58.904504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-877721ad-fc33-4725-b506-1e019045d450 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:58.904504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:58.904504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:58.904504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:58.904504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:58.904504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:58.904504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:58.904504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:58.904504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:58.904504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:58.904504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:58.904504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:58.904504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:45:58.904504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:45:58.904504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:45:58.904504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:45:58.905560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:58.905560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:58.905560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:58.905560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:58.905560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:58.905560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:58.905560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:58.905560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:58.905560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:58.905560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:58.905560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:58.905560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:58.905560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:58.905560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:58.905560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:58.905560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:58.905560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:45:58.906483 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:45:58.906483 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:45:58.906483 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:58.910643 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 534/1072] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:58.911561 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877721ad-fc33-4725-b506-1e019045d450 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:58.945645 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968909e-7c61-4bcc-8bdb-604a34670a32 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:58.983893 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1599f2f-2fc1-4471-8a6d-2d30a5fd06f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:58.984594 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1599f2f-2fc1-4471-8a6d-2d30a5fd06f7 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:58.984794 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1599f2f-2fc1-4471-8a6d-2d30a5fd06f7 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:58.985066 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1599f2f-2fc1-4471-8a6d-2d30a5fd06f7 None admin] PATH_INFO: `/v3/users/94f1fc3eb1ab4fe3a9386f6121033cca` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:58.986369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1599f2f-2fc1-4471-8a6d-2d30a5fd06f7 None admin] RBAC: Authorizing `identity:get_user(user_id=94f1fc3eb1ab4fe3a9386f6121033cca)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:58.988003 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1599f2f-2fc1-4471-8a6d-2d30a5fd06f7 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:58.989939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 539/1073] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:45:58 2022] GET /identity/v3/users/94f1fc3eb1ab4fe3a9386f6121033cca => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:58.996587 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1599f2f-2fc1-4471-8a6d-2d30a5fd06f7 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:59.005194 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:59.029645 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877721ad-fc33-4725-b506-1e019045d450 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:59.064834 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9798418c-3b93-42d1-b08e-82715504684d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:59.065445 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798418c-3b93-42d1-b08e-82715504684d None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:59.065723 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798418c-3b93-42d1-b08e-82715504684d None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:59.065989 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798418c-3b93-42d1-b08e-82715504684d None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:59.066657 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9798418c-3b93-42d1-b08e-82715504684d None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:59.066960 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9798418c-3b93-42d1-b08e-82715504684d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:59.068321 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9798418c-3b93-42d1-b08e-82715504684d None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:59.081646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 535/1074] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:45:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:45:59.082716 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d91a10-62f0-430c-9998-3268e3fe509c None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:59.082999 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d91a10-62f0-430c-9998-3268e3fe509c None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:59.083270 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d91a10-62f0-430c-9998-3268e3fe509c None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:59.084649 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 536/1075] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:45:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:45:59.087233 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d91a10-62f0-430c-9998-3268e3fe509c None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:59.111594 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:59.120841 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1599f2f-2fc1-4471-8a6d-2d30a5fd06f7 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:59.155810 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301d10a-64f0-40a1-8496-446b5971c733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:59.156455 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301d10a-64f0-40a1-8496-446b5971c733 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:59.156726 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301d10a-64f0-40a1-8496-446b5971c733 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:59.156998 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301d10a-64f0-40a1-8496-446b5971c733 None admin] PATH_INFO: `/v3/projects/85b7a190e444441c8361d1f925e0dcff` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:59.158129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7301d10a-64f0-40a1-8496-446b5971c733 None admin] RBAC: Authorizing `identity:get_project(project_id=85b7a190e444441c8361d1f925e0dcff)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:59.159604 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7301d10a-64f0-40a1-8496-446b5971c733 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:59.161248 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 540/1076] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:45:58 2022] GET /identity/v3/projects/85b7a190e444441c8361d1f925e0dcff => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:59.162382 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b39ef5-28c6-4629-95c6-4a2f856bcc6a None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:59.162680 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b39ef5-28c6-4629-95c6-4a2f856bcc6a None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:59.162952 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b39ef5-28c6-4629-95c6-4a2f856bcc6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:59.181762 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-c0b39ef5-28c6-4629-95c6-4a2f856bcc6a None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:45:59.183597 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-c0b39ef5-28c6-4629-95c6-4a2f856bcc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:45:59.193949 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d91a10-62f0-430c-9998-3268e3fe509c None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:59.229515 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd430b73-2547-49f2-a86a-29496773c17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:59.230058 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd430b73-2547-49f2-a86a-29496773c17f None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:59.230335 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd430b73-2547-49f2-a86a-29496773c17f None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:59.230635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd430b73-2547-49f2-a86a-29496773c17f None admin] PATH_INFO: `/v3/projects/service` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:59.246581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd430b73-2547-49f2-a86a-29496773c17f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:59.248031 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd430b73-2547-49f2-a86a-29496773c17f None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:59.249873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:59.252655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 541/1077] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:45:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:45:59.253467 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b39ef5-28c6-4629-95c6-4a2f856bcc6a None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:59.260804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-fd430b73-2547-49f2-a86a-29496773c17f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:59.260804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:45:59.260804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:45:59.260804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:45:59.260804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:45:59.260804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:45:59.260804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:45:59.260804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:45:59.260804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:45:59.260804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:45:59.260804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:45:59.260804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:45:59.260804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:45:59.260804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:45:59.260804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:45:59.267929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:45:59.267929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:45:59.267929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:45:59.267929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:45:59.267929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:45:59.267929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:45:59.267929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:45:59.267929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:45:59.267929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:45:59.267929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:45:59.267929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:45:59.267929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:45:59.267929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:45:59.267929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:45:59.267929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:45:59.267929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:45:59.267929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:45:59.268861 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:45:59.268861 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:45:59.268861 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:45:59.268861 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:45:59.268861 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:45:59.268861 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:45:59.268861 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:45:59.268861 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:45:59.268861 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 537/1078] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:45:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:45:59.268861 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd430b73-2547-49f2-a86a-29496773c17f None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:59.365030 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:59.374897 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b39ef5-28c6-4629-95c6-4a2f856bcc6a None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:59.407048 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd430b73-2547-49f2-a86a-29496773c17f None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:59.413527 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78940180-7ba0-450e-8472-18ee1fe7e1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:59.414606 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78940180-7ba0-450e-8472-18ee1fe7e1f7 None admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:59.414606 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78940180-7ba0-450e-8472-18ee1fe7e1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:59.414798 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78940180-7ba0-450e-8472-18ee1fe7e1f7 None admin] PATH_INFO: `/v3/projects/85b7a190e444441c8361d1f925e0dcff/users/94f1fc3eb1ab4fe3a9386f6121033cca/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:59.416708 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78940180-7ba0-450e-8472-18ee1fe7e1f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=85b7a190e444441c8361d1f925e0dcff, user_id=94f1fc3eb1ab4fe3a9386f6121033cca, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:59.418143 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78940180-7ba0-450e-8472-18ee1fe7e1f7 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:59.442038 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f60579-b033-4be3-a6d7-237e197b93a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:59.442678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f60579-b033-4be3-a6d7-237e197b93a3 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:59.442953 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f60579-b033-4be3-a6d7-237e197b93a3 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:59.443222 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f60579-b033-4be3-a6d7-237e197b93a3 None admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:59.444791 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f60579-b033-4be3-a6d7-237e197b93a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:45:59.444791 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f60579-b033-4be3-a6d7-237e197b93a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:59.445698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f60579-b033-4be3-a6d7-237e197b93a3 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:59.458736 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 538/1079] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:45:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:45:59.465394 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f60579-b033-4be3-a6d7-237e197b93a3 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:59.524099 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:59.551570 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 542/1080] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 31 10:45:59 2022] PUT /identity/v3/projects/85b7a190e444441c8361d1f925e0dcff/users/94f1fc3eb1ab4fe3a9386f6121033cca/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:45:59.562647 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78940180-7ba0-450e-8472-18ee1fe7e1f7 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:45:59.568298 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f60579-b033-4be3-a6d7-237e197b93a3 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:59.619418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2d9b3-7fe3-42b8-85ba-4e688e6e86ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:59.621712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2d9b3-7fe3-42b8-85ba-4e688e6e86ac None admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:59.622174 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2d9b3-7fe3-42b8-85ba-4e688e6e86ac None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:59.622528 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2d9b3-7fe3-42b8-85ba-4e688e6e86ac None admin] PATH_INFO: `/v3/projects/8db9b3f8a9f041938ac7b39003a35015/users/f11971a594014d86968c3356d11d42f7/roles/1715f4cc50354636b1f7c3a9864e1e80` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:59.625869 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2d9b3-7fe3-42b8-85ba-4e688e6e86ac None admin] RBAC: Authorizing `identity:create_grant(project_id=8db9b3f8a9f041938ac7b39003a35015, user_id=f11971a594014d86968c3356d11d42f7, role_id=1715f4cc50354636b1f7c3a9864e1e80)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:59.627803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2d9b3-7fe3-42b8-85ba-4e688e6e86ac None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:59.685073 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78940180-7ba0-450e-8472-18ee1fe7e1f7 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:45:59.732048 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d2baa-43ff-4b9d-baab-13654403ffcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:45:59.732656 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d2baa-43ff-4b9d-baab-13654403ffcc None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:45:59.732933 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d2baa-43ff-4b9d-baab-13654403ffcc None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:45:59.733209 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d2baa-43ff-4b9d-baab-13654403ffcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:45:59.752854 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:45:59.800951 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 539/1081] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 31 10:45:59 2022] PUT /identity/v3/projects/8db9b3f8a9f041938ac7b39003a35015/users/f11971a594014d86968c3356d11d42f7/roles/1715f4cc50354636b1f7c3a9864e1e80 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 10:45:59.804658 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d2baa-43ff-4b9d-baab-13654403ffcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:45:59.806162 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d2baa-43ff-4b9d-baab-13654403ffcc None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:45:59.933193 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 543/1082] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:45:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:45:59.939300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2d9b3-7fe3-42b8-85ba-4e688e6e86ac None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:00.052942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2d9b3-7fe3-42b8-85ba-4e688e6e86ac None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:00.071007 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:00.089634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd9dcd-2a3f-4924-b4e1-a5e93aa0fc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:00.090270 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd9dcd-2a3f-4924-b4e1-a5e93aa0fc4f None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:00.090596 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd9dcd-2a3f-4924-b4e1-a5e93aa0fc4f None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:00.090867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd9dcd-2a3f-4924-b4e1-a5e93aa0fc4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:00.103634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cd9dcd-2a3f-4924-b4e1-a5e93aa0fc4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:00.105913 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cd9dcd-2a3f-4924-b4e1-a5e93aa0fc4f None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:00.132200 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-e5cd9dcd-2a3f-4924-b4e1-a5e93aa0fc4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:46:00.132200 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:46:00.132200 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:46:00.132200 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:46:00.132200 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:46:00.132200 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:46:00.132200 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:46:00.132200 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:46:00.132200 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:46:00.132200 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:46:00.132200 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:46:00.132200 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:46:00.132200 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:46:00.132200 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:46:00.132200 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:46:00.132200 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:46:00.137448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:46:00.137448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:46:00.137448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:46:00.137448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:46:00.137448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:46:00.137448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:46:00.137448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:46:00.137448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:46:00.137448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:46:00.137448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:46:00.137448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:46:00.137448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:46:00.137448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:46:00.137448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:46:00.137448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:46:00.137448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:46:00.137448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:46:00.138505 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:46:00.138505 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:46:00.138505 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:46:00.138505 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 540/1083] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:45:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:46:00.140468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d2baa-43ff-4b9d-baab-13654403ffcc None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:00.194587 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:00.251889 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d2baa-43ff-4b9d-baab-13654403ffcc None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:00.288124 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0afe853-3dcf-4ee7-a666-7a7e69d62b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:00.288735 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0afe853-3dcf-4ee7-a666-7a7e69d62b26 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:00.289008 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0afe853-3dcf-4ee7-a666-7a7e69d62b26 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:00.289278 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0afe853-3dcf-4ee7-a666-7a7e69d62b26 None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:00.292749 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0afe853-3dcf-4ee7-a666-7a7e69d62b26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:46:00.292749 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0afe853-3dcf-4ee7-a666-7a7e69d62b26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:00.292749 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0afe853-3dcf-4ee7-a666-7a7e69d62b26 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:00.307652 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 544/1084] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:01.497531 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c18eb2-35db-40da-8078-b76468391c82 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:01.498104 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c18eb2-35db-40da-8078-b76468391c82 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:01.498104 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c18eb2-35db-40da-8078-b76468391c82 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:01.499447 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 541/1085] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:46:01.504594 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39246bd0-84cd-464d-bd2f-b46b45298585 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:01.505007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39246bd0-84cd-464d-bd2f-b46b45298585 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:01.505142 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39246bd0-84cd-464d-bd2f-b46b45298585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:01.523937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-39246bd0-84cd-464d-bd2f-b46b45298585 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:46:01.525120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-39246bd0-84cd-464d-bd2f-b46b45298585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:46:01.591666 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 545/1086] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:46:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:46:01.781427 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93f0ac-bcb8-4100-8cef-fb7784012855 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:01.781642 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93f0ac-bcb8-4100-8cef-fb7784012855 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:01.781888 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93f0ac-bcb8-4100-8cef-fb7784012855 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:01.783274 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 542/1087] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:46:01.790746 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1a36e-dc5f-4eb9-a5a3-3fb3f3b9faef None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:01.791029 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1a36e-dc5f-4eb9-a5a3-3fb3f3b9faef None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:01.791307 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1a36e-dc5f-4eb9-a5a3-3fb3f3b9faef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:01.810695 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-eaa1a36e-dc5f-4eb9-a5a3-3fb3f3b9faef None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:46:01.811883 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-eaa1a36e-dc5f-4eb9-a5a3-3fb3f3b9faef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:46:01.851718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:01.860418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93f0ac-bcb8-4100-8cef-fb7784012855 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:01.888909 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 546/1088] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:46:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:46:01.964169 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93f0ac-bcb8-4100-8cef-fb7784012855 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:01.999761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b6dc7-bc37-44af-8c24-edc505bc691f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:02.000393 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b6dc7-bc37-44af-8c24-edc505bc691f None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:02.000668 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b6dc7-bc37-44af-8c24-edc505bc691f None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:02.000940 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b6dc7-bc37-44af-8c24-edc505bc691f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:02.011329 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b6dc7-bc37-44af-8c24-edc505bc691f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:46:02.016105 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-7c7b6dc7-bc37-44af-8c24-edc505bc691f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:46:02.016105 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:46:02.016105 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:46:02.016105 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:46:02.016105 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:46:02.016105 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:46:02.016105 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:46:02.016105 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:46:02.016105 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:46:02.016105 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:46:02.016105 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:46:02.016105 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:46:02.016105 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:46:02.016105 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:46:02.016105 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:46:02.016105 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:46:02.017247 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:46:02.017247 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:46:02.017247 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:46:02.017247 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:46:02.017247 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:46:02.017247 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:46:02.017247 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:46:02.017247 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:46:02.017247 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:46:02.017247 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:46:02.017247 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:46:02.017247 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:46:02.017247 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:46:02.017247 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:46:02.017247 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:46:02.017247 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:46:02.018120 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:46:02.018120 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:46:02.018120 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:46:02.018120 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:46:02.018120 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:46:02.018120 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:46:02.018120 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:46:02.018120 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:46:02.018120 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:46:02.018120 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:46:02.018120 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:46:02.018120 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:46:02.018120 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 543/1089] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:46:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:46:02.023408 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1a36e-dc5f-4eb9-a5a3-3fb3f3b9faef None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:02.137865 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1a36e-dc5f-4eb9-a5a3-3fb3f3b9faef None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:02.168220 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b6dc7-bc37-44af-8c24-edc505bc691f None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:02.172377 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ba883-db3e-4200-a80e-24b2f4460f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:02.172990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ba883-db3e-4200-a80e-24b2f4460f66 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:02.173267 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ba883-db3e-4200-a80e-24b2f4460f66 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:02.173541 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ba883-db3e-4200-a80e-24b2f4460f66 None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:02.174237 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ba883-db3e-4200-a80e-24b2f4460f66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:46:02.174568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ba883-db3e-4200-a80e-24b2f4460f66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:02.175931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ba883-db3e-4200-a80e-24b2f4460f66 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:02.184626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 547/1090] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:46:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:02.190164 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ba883-db3e-4200-a80e-24b2f4460f66 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:02.200138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:02.223700 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:02.271933 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b6dc7-bc37-44af-8c24-edc505bc691f None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:02.293572 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ba883-db3e-4200-a80e-24b2f4460f66 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:02.306408 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795da5d1-a17b-4ca0-be91-c5b14c77df90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:02.307081 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795da5d1-a17b-4ca0-be91-c5b14c77df90 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:02.307352 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795da5d1-a17b-4ca0-be91-c5b14c77df90 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:02.307618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795da5d1-a17b-4ca0-be91-c5b14c77df90 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:02.317679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795da5d1-a17b-4ca0-be91-c5b14c77df90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:46:02.322411 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-795da5d1-a17b-4ca0-be91-c5b14c77df90 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 10:46:02.322411 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:46:02.322411 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:46:02.322411 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:46:02.322411 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:46:02.322411 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:46:02.322411 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:46:02.322411 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:46:02.322411 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:46:02.322411 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:46:02.322411 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:46:02.322411 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:46:02.322411 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:46:02.322411 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:46:02.322411 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:46:02.322411 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:46:02.324828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:46:02.324828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:46:02.324828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:46:02.324828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:46:02.324828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:46:02.324828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:46:02.324828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:46:02.324828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:46:02.324828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:46:02.324828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:46:02.324828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:46:02.324828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:46:02.324828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:46:02.324828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:46:02.324828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:46:02.324828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:46:02.325873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:46:02.325873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:46:02.325873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:46:02.325873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:46:02.325873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:46:02.325873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:46:02.325873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:46:02.325873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:46:02.325873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:46:02.325873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:46:02.325873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 10:46:02.325873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:46:02.325873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 544/1091] 10.222.0.16 () {56 vars in 1120 bytes} [Wed Aug 31 10:46:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:46:02.331142 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795da5d1-a17b-4ca0-be91-c5b14c77df90 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:02.331333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180d5d5-21dc-482a-91e8-7388dc135da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:02.332055 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180d5d5-21dc-482a-91e8-7388dc135da2 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:02.332325 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180d5d5-21dc-482a-91e8-7388dc135da2 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:02.332587 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180d5d5-21dc-482a-91e8-7388dc135da2 None admin] PATH_INFO: `/v3/users/94f1fc3eb1ab4fe3a9386f6121033cca` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:02.333732 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b180d5d5-21dc-482a-91e8-7388dc135da2 None admin] RBAC: Authorizing `identity:get_user(user_id=94f1fc3eb1ab4fe3a9386f6121033cca)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:02.335203 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b180d5d5-21dc-482a-91e8-7388dc135da2 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:02.342545 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 548/1092] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:46:02 2022] GET /identity/v3/users/94f1fc3eb1ab4fe3a9386f6121033cca => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:46:02.343686 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180d5d5-21dc-482a-91e8-7388dc135da2 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:02.430472 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:02.440173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795da5d1-a17b-4ca0-be91-c5b14c77df90 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:02.443731 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:02.446358 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180d5d5-21dc-482a-91e8-7388dc135da2 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:02.477695 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1407e-bcf8-43cd-a684-83f2d071c4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:02.478372 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1407e-bcf8-43cd-a684-83f2d071c4db None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:02.483803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1407e-bcf8-43cd-a684-83f2d071c4db None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:02.483803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1407e-bcf8-43cd-a684-83f2d071c4db None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:02.483803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e1407e-bcf8-43cd-a684-83f2d071c4db None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:46:02.483803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e1407e-bcf8-43cd-a684-83f2d071c4db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:02.483803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e1407e-bcf8-43cd-a684-83f2d071c4db None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:02.484296 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4cb72-0dc8-4429-8a8a-30e20e95b1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:02.484296 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4cb72-0dc8-4429-8a8a-30e20e95b1cb None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:02.484296 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4cb72-0dc8-4429-8a8a-30e20e95b1cb None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:02.484296 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4cb72-0dc8-4429-8a8a-30e20e95b1cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:02.489795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 545/1093] 10.222.0.16 () {56 vars in 1115 bytes} [Wed Aug 31 10:46:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:46:02.496634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1407e-bcf8-43cd-a684-83f2d071c4db None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:02.554961 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a4cb72-0dc8-4429-8a8a-30e20e95b1cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:02.556421 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a4cb72-0dc8-4429-8a8a-30e20e95b1cb None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:02.600563 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1407e-bcf8-43cd-a684-83f2d071c4db None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:02.635789 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0df4e-7d81-43e9-930e-d44228f58d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:02.636409 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0df4e-7d81-43e9-930e-d44228f58d29 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:02.636686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0df4e-7d81-43e9-930e-d44228f58d29 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:02.636957 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0df4e-7d81-43e9-930e-d44228f58d29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:02.647953 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc0df4e-7d81-43e9-930e-d44228f58d29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:02.649409 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc0df4e-7d81-43e9-930e-d44228f58d29 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:02.655433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:02.659211 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 549/1094] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:46:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:46:02.664751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-bfc0df4e-7d81-43e9-930e-d44228f58d29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:46:02.664751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:46:02.664751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:46:02.664751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:46:02.664751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:46:02.664751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:46:02.664751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:46:02.664751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:46:02.664751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:46:02.664751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:46:02.664751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:46:02.664751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:46:02.664751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:46:02.664751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:46:02.664751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:46:02.664751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:46:02.671232 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:46:02.671232 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:46:02.671232 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:46:02.671232 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:46:02.671232 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:46:02.671232 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:46:02.671232 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:46:02.671232 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:46:02.671232 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:46:02.671232 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:46:02.671232 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:46:02.671232 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:46:02.671232 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:46:02.671232 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:46:02.671232 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:46:02.671232 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:46:02.671232 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:46:02.672165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:46:02.672165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:46:02.672165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:46:02.672333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4cb72-0dc8-4429-8a8a-30e20e95b1cb None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:02.672406 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 546/1095] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:46:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:46:02.673511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0df4e-7d81-43e9-930e-d44228f58d29 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:02.677759 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:02.765524 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4cb72-0dc8-4429-8a8a-30e20e95b1cb None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:02.781499 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0df4e-7d81-43e9-930e-d44228f58d29 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:02.801620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480cd72e-a1e7-47f3-be58-a8df3ecf7276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:02.802243 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480cd72e-a1e7-47f3-be58-a8df3ecf7276 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:02.802547 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480cd72e-a1e7-47f3-be58-a8df3ecf7276 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:02.802817 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480cd72e-a1e7-47f3-be58-a8df3ecf7276 None admin] PATH_INFO: `/v3/projects/85b7a190e444441c8361d1f925e0dcff` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:02.803931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480cd72e-a1e7-47f3-be58-a8df3ecf7276 None admin] RBAC: Authorizing `identity:get_project(project_id=85b7a190e444441c8361d1f925e0dcff)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:02.805355 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480cd72e-a1e7-47f3-be58-a8df3ecf7276 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:02.807459 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 550/1096] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:46:02 2022] GET /identity/v3/projects/85b7a190e444441c8361d1f925e0dcff => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:46:02.812986 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480cd72e-a1e7-47f3-be58-a8df3ecf7276 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:02.815419 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9eb1b1-0ef2-46d6-b0fa-2cbdedd84ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:02.816033 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9eb1b1-0ef2-46d6-b0fa-2cbdedd84ad6 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:02.816308 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9eb1b1-0ef2-46d6-b0fa-2cbdedd84ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:02.816582 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9eb1b1-0ef2-46d6-b0fa-2cbdedd84ad6 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:02.817229 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9eb1b1-0ef2-46d6-b0fa-2cbdedd84ad6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:46:02.817529 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9eb1b1-0ef2-46d6-b0fa-2cbdedd84ad6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:02.818926 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9eb1b1-0ef2-46d6-b0fa-2cbdedd84ad6 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:02.831050 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 547/1097] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:02.836887 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9eb1b1-0ef2-46d6-b0fa-2cbdedd84ad6 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:02.882450 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:02.902801 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:02.918910 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480cd72e-a1e7-47f3-be58-a8df3ecf7276 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:02.942009 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9eb1b1-0ef2-46d6-b0fa-2cbdedd84ad6 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:02.952795 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd580655-fb29-4470-aa69-d7cf01186775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:02.953405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd580655-fb29-4470-aa69-d7cf01186775 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:02.953677 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd580655-fb29-4470-aa69-d7cf01186775 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:02.953942 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd580655-fb29-4470-aa69-d7cf01186775 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:02.954772 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd580655-fb29-4470-aa69-d7cf01186775 None admin] RBAC: Adding query filter params (role.id=13ca33d694c645d6920930d39dc52a0d, scope.project.id=85b7a190e444441c8361d1f925e0dcff, user.id=94f1fc3eb1ab4fe3a9386f6121033cca) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:46:02.955077 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd580655-fb29-4470-aa69-d7cf01186775 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:02.956492 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd580655-fb29-4470-aa69-d7cf01186775 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:02.962323 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 551/1098] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:46:02 2022] GET /identity/v3/role_assignments?user.id=94f1fc3eb1ab4fe3a9386f6121033cca&scope.project.id=85b7a190e444441c8361d1f925e0dcff&role.id=13ca33d694c645d6920930d39dc52a0d => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:46:02.984059 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b37fdd-2b48-4e71-83d1-7f106a96b76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:02.984721 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b37fdd-2b48-4e71-83d1-7f106a96b76e None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:02.984996 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b37fdd-2b48-4e71-83d1-7f106a96b76e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:02.985268 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b37fdd-2b48-4e71-83d1-7f106a96b76e None admin] PATH_INFO: `/v3/users/nova` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:02.993839 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b37fdd-2b48-4e71-83d1-7f106a96b76e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:02.995389 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b37fdd-2b48-4e71-83d1-7f106a96b76e None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:03.008278 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-06b37fdd-2b48-4e71-83d1-7f106a96b76e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:46:03.008278 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:46:03.008278 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:46:03.008278 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:46:03.008278 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:46:03.008278 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:46:03.008278 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:46:03.008278 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:46:03.008278 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:46:03.008278 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:46:03.008278 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:46:03.008278 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:46:03.008278 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:46:03.008278 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:46:03.008278 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:46:03.008278 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:46:03.009485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:46:03.009485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:46:03.009485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:46:03.009485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:46:03.009485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:46:03.009485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:46:03.009485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:46:03.009485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:46:03.009485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:46:03.009485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:46:03.009485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:46:03.009485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:46:03.009485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:46:03.009485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:46:03.009485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:46:03.009485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:46:03.009485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:46:03.009485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:46:03.011125 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:46:03.011125 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:46:03.011125 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:46:03.011125 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:46:03.011125 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:46:03.011125 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:46:03.011125 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:46:03.011125 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:46:03.011125 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:46:03.011125 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:46:03.011125 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 548/1099] 10.222.0.16 () {56 vars in 1093 bytes} [Wed Aug 31 10:46:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:46:03.016347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd580655-fb29-4470-aa69-d7cf01186775 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:03.130569 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd580655-fb29-4470-aa69-d7cf01186775 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:03.160875 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:03.165884 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7aaef-fc6a-4e1e-b66a-386eef79310e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:03.167134 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7aaef-fc6a-4e1e-b66a-386eef79310e None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:03.167410 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7aaef-fc6a-4e1e-b66a-386eef79310e None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:03.167683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7aaef-fc6a-4e1e-b66a-386eef79310e None admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:03.168415 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e7aaef-fc6a-4e1e-b66a-386eef79310e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:46:03.168717 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e7aaef-fc6a-4e1e-b66a-386eef79310e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:03.170221 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e7aaef-fc6a-4e1e-b66a-386eef79310e None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:03.189311 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 552/1100] 10.222.0.16 () {56 vars in 1133 bytes} [Wed Aug 31 10:46:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:46:03.195179 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b37fdd-2b48-4e71-83d1-7f106a96b76e None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:03.297417 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b37fdd-2b48-4e71-83d1-7f106a96b76e None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:03.320989 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:03.331937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc153b53-246c-4b51-b3b3-62ae3eb176c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:03.333790 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc153b53-246c-4b51-b3b3-62ae3eb176c8 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:03.333790 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc153b53-246c-4b51-b3b3-62ae3eb176c8 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:03.333790 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc153b53-246c-4b51-b3b3-62ae3eb176c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:03.402814 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc153b53-246c-4b51-b3b3-62ae3eb176c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:03.404266 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc153b53-246c-4b51-b3b3-62ae3eb176c8 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:03.504617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 549/1101] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:46:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:46:03.510556 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7aaef-fc6a-4e1e-b66a-386eef79310e None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:03.631887 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7aaef-fc6a-4e1e-b66a-386eef79310e None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:03.667160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3ac08-af33-43aa-8738-58399bc8b432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:03.667794 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3ac08-af33-43aa-8738-58399bc8b432 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:03.668073 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3ac08-af33-43aa-8738-58399bc8b432 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:03.668347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3ac08-af33-43aa-8738-58399bc8b432 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:03.678767 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e3ac08-af33-43aa-8738-58399bc8b432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:03.680261 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e3ac08-af33-43aa-8738-58399bc8b432 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:03.695715 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-65e3ac08-af33-43aa-8738-58399bc8b432 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:46:03.695715 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:46:03.695715 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:46:03.695715 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:46:03.695715 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:46:03.695715 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:46:03.695715 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:46:03.695715 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:46:03.695715 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:46:03.695715 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:46:03.695715 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:46:03.695715 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:46:03.695715 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:46:03.695715 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:46:03.695715 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:46:03.695715 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:46:03.696866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:46:03.696866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:46:03.696866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:46:03.696866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:46:03.696866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:46:03.696866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:46:03.696866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:46:03.696866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:46:03.696866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:46:03.696866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:46:03.696866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:46:03.696866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:46:03.696866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:46:03.696866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:46:03.696866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:46:03.696866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:46:03.696866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:46:03.699300 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:46:03.699300 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:46:03.699300 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:46:03.702485 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 553/1102] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:46:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:46:03.703491 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc153b53-246c-4b51-b3b3-62ae3eb176c8 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:03.728304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:03.805730 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc153b53-246c-4b51-b3b3-62ae3eb176c8 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:03.841223 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e703d04-53c9-46b8-b66e-90ca072da210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:03.841833 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e703d04-53c9-46b8-b66e-90ca072da210 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:03.842112 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e703d04-53c9-46b8-b66e-90ca072da210 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:03.842549 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e703d04-53c9-46b8-b66e-90ca072da210 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:03.843055 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e703d04-53c9-46b8-b66e-90ca072da210 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:46:03.843362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e703d04-53c9-46b8-b66e-90ca072da210 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:03.844736 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e703d04-53c9-46b8-b66e-90ca072da210 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:03.856404 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 550/1103] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:03.862116 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3ac08-af33-43aa-8738-58399bc8b432 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:03.906310 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:03.964123 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3ac08-af33-43aa-8738-58399bc8b432 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:03.998755 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd20669-42a8-4780-acc0-489ce0be0b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:03.999387 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd20669-42a8-4780-acc0-489ce0be0b5a None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:03.999662 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd20669-42a8-4780-acc0-489ce0be0b5a None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:03.999930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd20669-42a8-4780-acc0-489ce0be0b5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:04.006767 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd20669-42a8-4780-acc0-489ce0be0b5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:04.008216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd20669-42a8-4780-acc0-489ce0be0b5a None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:04.020082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-5cd20669-42a8-4780-acc0-489ce0be0b5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:46:04.020082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:46:04.020082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:46:04.020082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:46:04.020082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:46:04.020082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:46:04.020082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:46:04.020082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:46:04.020082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:46:04.020082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:46:04.020082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:46:04.020082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:46:04.020082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:46:04.020082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:46:04.020082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:46:04.021120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:46:04.021120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:46:04.021120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:46:04.021120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:46:04.021120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:46:04.021120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:46:04.021120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:46:04.021120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:46:04.021120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:46:04.021120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:46:04.021120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:46:04.021120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:46:04.021120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:46:04.021120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:46:04.021120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:46:04.021120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:46:04.021120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:46:04.022339 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:46:04.022339 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:46:04.022339 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:46:04.022339 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:46:04.022339 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:46:04.022339 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:46:04.022339 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:46:04.022339 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:46:04.022339 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 554/1104] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:46:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:46:04.027794 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e703d04-53c9-46b8-b66e-90ca072da210 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:04.116950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:04.133506 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e703d04-53c9-46b8-b66e-90ca072da210 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:04.175240 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992fe74c-47d2-4c7b-8e09-dd3e853330e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:04.175240 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992fe74c-47d2-4c7b-8e09-dd3e853330e3 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:04.175240 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992fe74c-47d2-4c7b-8e09-dd3e853330e3 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:04.175240 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992fe74c-47d2-4c7b-8e09-dd3e853330e3 None admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:04.175683 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992fe74c-47d2-4c7b-8e09-dd3e853330e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:46:04.175683 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992fe74c-47d2-4c7b-8e09-dd3e853330e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:04.175683 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992fe74c-47d2-4c7b-8e09-dd3e853330e3 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:04.190782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 551/1105] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:46:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:04.192017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd20669-42a8-4780-acc0-489ce0be0b5a None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:04.297807 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd20669-42a8-4780-acc0-489ce0be0b5a None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:04.299465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:04.332984 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63f49a2-81eb-4204-9aa7-7655bd5c0a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:04.333595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f49a2-81eb-4204-9aa7-7655bd5c0a90 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:04.333866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f49a2-81eb-4204-9aa7-7655bd5c0a90 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:04.334134 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f49a2-81eb-4204-9aa7-7655bd5c0a90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:04.334908 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63f49a2-81eb-4204-9aa7-7655bd5c0a90 None admin] RBAC: Adding query filter params (role.id=1715f4cc50354636b1f7c3a9864e1e80, scope.project.id=8db9b3f8a9f041938ac7b39003a35015, user.id=f11971a594014d86968c3356d11d42f7) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:46:04.335212 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63f49a2-81eb-4204-9aa7-7655bd5c0a90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:04.337163 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63f49a2-81eb-4204-9aa7-7655bd5c0a90 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:04.345073 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 555/1106] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:46:04 2022] GET /identity/v3/role_assignments?user.id=f11971a594014d86968c3356d11d42f7&scope.project.id=8db9b3f8a9f041938ac7b39003a35015&role.id=1715f4cc50354636b1f7c3a9864e1e80 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:04.918747 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6276d-774e-48ee-89db-3f7431a09d5c None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:04.919299 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6276d-774e-48ee-89db-3f7431a09d5c None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:04.919299 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6276d-774e-48ee-89db-3f7431a09d5c None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:04.920779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 552/1107] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:46:04.925927 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7db0a1-d81c-4494-9865-d17dd2f589ed None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:04.926369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7db0a1-d81c-4494-9865-d17dd2f589ed None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:04.926486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7db0a1-d81c-4494-9865-d17dd2f589ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:04.945216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-2a7db0a1-d81c-4494-9865-d17dd2f589ed None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:46:04.946447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2a7db0a1-d81c-4494-9865-d17dd2f589ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:46:05.015084 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 556/1108] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:46:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:46:05.283725 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6276d-774e-48ee-89db-3f7431a09d5c None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:05.400455 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6276d-774e-48ee-89db-3f7431a09d5c None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:05.434672 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ad4d4-7803-4ac9-bbab-48b15bb796e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:05.435304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ad4d4-7803-4ac9-bbab-48b15bb796e4 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:05.435577 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ad4d4-7803-4ac9-bbab-48b15bb796e4 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:05.435844 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ad4d4-7803-4ac9-bbab-48b15bb796e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:05.436950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ad4d4-7803-4ac9-bbab-48b15bb796e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:05.438401 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ad4d4-7803-4ac9-bbab-48b15bb796e4 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:05.439984 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 553/1109] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:46:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:05.445754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7db0a1-d81c-4494-9865-d17dd2f589ed None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:05.538115 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:05.543785 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7db0a1-d81c-4494-9865-d17dd2f589ed None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:05.577015 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25998697-e404-47aa-a247-2b1dc64c1ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:05.577618 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25998697-e404-47aa-a247-2b1dc64c1ad1 None admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:05.577898 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25998697-e404-47aa-a247-2b1dc64c1ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:05.578372 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25998697-e404-47aa-a247-2b1dc64c1ad1 None admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:05.579608 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25998697-e404-47aa-a247-2b1dc64c1ad1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:05.581088 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25998697-e404-47aa-a247-2b1dc64c1ad1 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:05.687910 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 557/1110] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 31 10:46:05 2022] POST /identity/v3/users => generated 286 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:46:07.685576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ed561-ac53-4e9a-ad5c-adc98101e089 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:07.686123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ed561-ac53-4e9a-ad5c-adc98101e089 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:07.686123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ed561-ac53-4e9a-ad5c-adc98101e089 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:07.687495 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 554/1111] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:46:07.692750 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96584b2a-b63a-4ed1-9aea-f3d0608ccc6c None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:07.693182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96584b2a-b63a-4ed1-9aea-f3d0608ccc6c None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:07.693298 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96584b2a-b63a-4ed1-9aea-f3d0608ccc6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:07.712502 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-96584b2a-b63a-4ed1-9aea-f3d0608ccc6c None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:46:07.714857 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-96584b2a-b63a-4ed1-9aea-f3d0608ccc6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:46:07.782104 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 558/1112] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:46:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:46:08.048194 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ed561-ac53-4e9a-ad5c-adc98101e089 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:08.063666 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:08.150578 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ed561-ac53-4e9a-ad5c-adc98101e089 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:08.185018 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc501b3-cb76-4b23-a84f-8a78b62c09d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:08.185639 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc501b3-cb76-4b23-a84f-8a78b62c09d1 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:08.185918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc501b3-cb76-4b23-a84f-8a78b62c09d1 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:08.186188 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc501b3-cb76-4b23-a84f-8a78b62c09d1 None admin] PATH_INFO: `/v3/roles/31d4b1dc082c41f7b6f18c338e7b51df` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:08.187775 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc501b3-cb76-4b23-a84f-8a78b62c09d1 None admin] RBAC: Authorizing `identity:get_role(role_id=31d4b1dc082c41f7b6f18c338e7b51df)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:08.189138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc501b3-cb76-4b23-a84f-8a78b62c09d1 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:08.190489 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 555/1113] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:46:08 2022] GET /identity/v3/roles/31d4b1dc082c41f7b6f18c338e7b51df => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:08.196295 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96584b2a-b63a-4ed1-9aea-f3d0608ccc6c None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:08.273980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:08.291595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96584b2a-b63a-4ed1-9aea-f3d0608ccc6c None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:08.324942 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5440c-efc1-4cbf-8c8c-9b7e213f5f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:08.325499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5440c-efc1-4cbf-8c8c-9b7e213f5f21 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:08.325773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5440c-efc1-4cbf-8c8c-9b7e213f5f21 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:08.326042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5440c-efc1-4cbf-8c8c-9b7e213f5f21 None admin] PATH_INFO: `/v3/users/042afe31291a46d19f7155c3a7b637b7` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:08.338619 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b5440c-efc1-4cbf-8c8c-9b7e213f5f21 None admin] RBAC: Authorizing `identity:get_user(user_id=042afe31291a46d19f7155c3a7b637b7)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:08.340046 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b5440c-efc1-4cbf-8c8c-9b7e213f5f21 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:08.341621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 559/1114] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:46:08 2022] GET /identity/v3/users/042afe31291a46d19f7155c3a7b637b7 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:08.347515 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc501b3-cb76-4b23-a84f-8a78b62c09d1 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:08.427837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:08.446817 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc501b3-cb76-4b23-a84f-8a78b62c09d1 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:08.481164 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09959c1a-116c-4e07-8f4e-7890a540b509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:08.481763 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09959c1a-116c-4e07-8f4e-7890a540b509 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:08.482036 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09959c1a-116c-4e07-8f4e-7890a540b509 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:08.482304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09959c1a-116c-4e07-8f4e-7890a540b509 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:08.550245 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09959c1a-116c-4e07-8f4e-7890a540b509 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:08.551721 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09959c1a-116c-4e07-8f4e-7890a540b509 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:08.632259 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:08.652543 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 556/1115] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:46:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:46:08.658661 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5440c-efc1-4cbf-8c8c-9b7e213f5f21 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:08.761541 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5440c-efc1-4cbf-8c8c-9b7e213f5f21 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:08.796046 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c282de1-fde8-4c13-877a-774c04d29157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:08.796665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c282de1-fde8-4c13-877a-774c04d29157 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:08.796938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c282de1-fde8-4c13-877a-774c04d29157 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:08.797201 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c282de1-fde8-4c13-877a-774c04d29157 None admin] PATH_INFO: `/v3/projects/85b7a190e444441c8361d1f925e0dcff` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:08.798297 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c282de1-fde8-4c13-877a-774c04d29157 None admin] RBAC: Authorizing `identity:get_project(project_id=85b7a190e444441c8361d1f925e0dcff)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:08.799744 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c282de1-fde8-4c13-877a-774c04d29157 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:08.801346 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 560/1116] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:46:08 2022] GET /identity/v3/projects/85b7a190e444441c8361d1f925e0dcff => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:08.807761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09959c1a-116c-4e07-8f4e-7890a540b509 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:08.903798 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09959c1a-116c-4e07-8f4e-7890a540b509 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:08.936207 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f7dd4-c6a5-4b6c-a7dc-d2ac220e1dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:08.936791 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f7dd4-c6a5-4b6c-a7dc-d2ac220e1dda None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:08.937063 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f7dd4-c6a5-4b6c-a7dc-d2ac220e1dda None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:08.937328 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f7dd4-c6a5-4b6c-a7dc-d2ac220e1dda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:08.938042 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f7dd4-c6a5-4b6c-a7dc-d2ac220e1dda None admin] RBAC: Adding query filter params (role.id=31d4b1dc082c41f7b6f18c338e7b51df, scope.project.id=85b7a190e444441c8361d1f925e0dcff, user.id=042afe31291a46d19f7155c3a7b637b7) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:46:08.938343 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f7dd4-c6a5-4b6c-a7dc-d2ac220e1dda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:08.939775 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f7dd4-c6a5-4b6c-a7dc-d2ac220e1dda None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:08.944973 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 557/1117] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:46:08 2022] GET /identity/v3/role_assignments?user.id=042afe31291a46d19f7155c3a7b637b7&scope.project.id=85b7a190e444441c8361d1f925e0dcff&role.id=31d4b1dc082c41f7b6f18c338e7b51df => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:10.846594 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9a248-f497-48e5-9cfb-68e76feac105 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:10.847152 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9a248-f497-48e5-9cfb-68e76feac105 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:10.847152 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9a248-f497-48e5-9cfb-68e76feac105 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:10.848475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 561/1118] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:46:10.853651 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d59da-8162-4785-95b0-0f72ba699f31 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:10.854070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d59da-8162-4785-95b0-0f72ba699f31 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:10.854229 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d59da-8162-4785-95b0-0f72ba699f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:10.873555 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-150d59da-8162-4785-95b0-0f72ba699f31 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:46:10.874773 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-150d59da-8162-4785-95b0-0f72ba699f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:46:10.913226 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:10.940923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 558/1119] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:46:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:46:11.209119 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9a248-f497-48e5-9cfb-68e76feac105 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:11.220645 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:11.310516 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9a248-f497-48e5-9cfb-68e76feac105 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:11.343821 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af6219-a09d-4674-bf90-e086ff9dd4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:11.344433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af6219-a09d-4674-bf90-e086ff9dd4fd None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:11.344707 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af6219-a09d-4674-bf90-e086ff9dd4fd None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:11.344975 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af6219-a09d-4674-bf90-e086ff9dd4fd None admin] PATH_INFO: `/v3/roles/31d4b1dc082c41f7b6f18c338e7b51df` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:11.346341 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0af6219-a09d-4674-bf90-e086ff9dd4fd None admin] RBAC: Authorizing `identity:get_role(role_id=31d4b1dc082c41f7b6f18c338e7b51df)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:11.347716 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0af6219-a09d-4674-bf90-e086ff9dd4fd None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:11.349014 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 562/1120] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:46:11 2022] GET /identity/v3/roles/31d4b1dc082c41f7b6f18c338e7b51df => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:11.355173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d59da-8162-4785-95b0-0f72ba699f31 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:11.450521 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d59da-8162-4785-95b0-0f72ba699f31 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:11.483674 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccab15d-9a2e-4ca5-a910-5f22c2a607dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:11.484297 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccab15d-9a2e-4ca5-a910-5f22c2a607dd None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:11.484668 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccab15d-9a2e-4ca5-a910-5f22c2a607dd None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:11.484936 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccab15d-9a2e-4ca5-a910-5f22c2a607dd None admin] PATH_INFO: `/v3/users/042afe31291a46d19f7155c3a7b637b7` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:11.486065 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccab15d-9a2e-4ca5-a910-5f22c2a607dd None admin] RBAC: Authorizing `identity:get_user(user_id=042afe31291a46d19f7155c3a7b637b7)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:11.487508 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccab15d-9a2e-4ca5-a910-5f22c2a607dd None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:11.489096 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 559/1121] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:46:11 2022] GET /identity/v3/users/042afe31291a46d19f7155c3a7b637b7 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:46:11.495879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af6219-a09d-4674-bf90-e086ff9dd4fd None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:11.570615 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:11.594208 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af6219-a09d-4674-bf90-e086ff9dd4fd None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:11.628547 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a9039-a355-4203-a3b7-a69666e614a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:11.629168 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a9039-a355-4203-a3b7-a69666e614a2 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:11.629441 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a9039-a355-4203-a3b7-a69666e614a2 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:11.629709 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a9039-a355-4203-a3b7-a69666e614a2 None admin] PATH_INFO: `/v3/projects/85b7a190e444441c8361d1f925e0dcff` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:11.630847 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54a9039-a355-4203-a3b7-a69666e614a2 None admin] RBAC: Authorizing `identity:get_project(project_id=85b7a190e444441c8361d1f925e0dcff)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:11.632269 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54a9039-a355-4203-a3b7-a69666e614a2 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:11.633879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 563/1122] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:46:11 2022] GET /identity/v3/projects/85b7a190e444441c8361d1f925e0dcff => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:11.640574 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccab15d-9a2e-4ca5-a910-5f22c2a607dd None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:11.683597 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:11.738858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccab15d-9a2e-4ca5-a910-5f22c2a607dd None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:11.772637 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913934d-9a56-48c9-9ae0-c763a366faf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:11.773280 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913934d-9a56-48c9-9ae0-c763a366faf1 None admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:11.773550 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913934d-9a56-48c9-9ae0-c763a366faf1 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:11.773814 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913934d-9a56-48c9-9ae0-c763a366faf1 None admin] PATH_INFO: `/v3/projects/85b7a190e444441c8361d1f925e0dcff/users/042afe31291a46d19f7155c3a7b637b7/roles/31d4b1dc082c41f7b6f18c338e7b51df` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:11.776234 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913934d-9a56-48c9-9ae0-c763a366faf1 None admin] RBAC: Authorizing `identity:create_grant(project_id=85b7a190e444441c8361d1f925e0dcff, user_id=042afe31291a46d19f7155c3a7b637b7, role_id=31d4b1dc082c41f7b6f18c338e7b51df)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:11.777652 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913934d-9a56-48c9-9ae0-c763a366faf1 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:11.847425 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 560/1123] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 31 10:46:11 2022] PUT /identity/v3/projects/85b7a190e444441c8361d1f925e0dcff/users/042afe31291a46d19f7155c3a7b637b7/roles/31d4b1dc082c41f7b6f18c338e7b51df => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:46:13.720339 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef8f3d-88f7-4433-8d4b-a8026ffdb526 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:13.720872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef8f3d-88f7-4433-8d4b-a8026ffdb526 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:13.720872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef8f3d-88f7-4433-8d4b-a8026ffdb526 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:13.722198 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 564/1124] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:46:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:46:13.727191 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce529c0-2363-41f9-8a92-6724f835211a None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:13.727600 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce529c0-2363-41f9-8a92-6724f835211a None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:13.727766 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce529c0-2363-41f9-8a92-6724f835211a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:13.746574 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-1ce529c0-2363-41f9-8a92-6724f835211a None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:46:13.747752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-1ce529c0-2363-41f9-8a92-6724f835211a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:46:13.812453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 561/1125] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:46:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:46:14.084470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef8f3d-88f7-4433-8d4b-a8026ffdb526 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:14.191545 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef8f3d-88f7-4433-8d4b-a8026ffdb526 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:14.225351 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13874020-d57c-413b-82a5-1a075b9977bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:14.225959 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13874020-d57c-413b-82a5-1a075b9977bb None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:14.226232 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13874020-d57c-413b-82a5-1a075b9977bb None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:14.226532 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13874020-d57c-413b-82a5-1a075b9977bb None admin] PATH_INFO: `/v3/roles/31d4b1dc082c41f7b6f18c338e7b51df` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:14.227888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13874020-d57c-413b-82a5-1a075b9977bb None admin] RBAC: Authorizing `identity:get_role(role_id=31d4b1dc082c41f7b6f18c338e7b51df)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:14.229297 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13874020-d57c-413b-82a5-1a075b9977bb None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:14.230630 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 565/1126] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:46:14 2022] GET /identity/v3/roles/31d4b1dc082c41f7b6f18c338e7b51df => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:14.236461 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce529c0-2363-41f9-8a92-6724f835211a None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:14.266870 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:14.333706 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce529c0-2363-41f9-8a92-6724f835211a None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:14.367884 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888316cb-2341-4a5e-83ba-e438dfaf90a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:14.368702 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888316cb-2341-4a5e-83ba-e438dfaf90a9 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:14.368975 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888316cb-2341-4a5e-83ba-e438dfaf90a9 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:14.369750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888316cb-2341-4a5e-83ba-e438dfaf90a9 None admin] PATH_INFO: `/v3/users/042afe31291a46d19f7155c3a7b637b7` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:14.370914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888316cb-2341-4a5e-83ba-e438dfaf90a9 None admin] RBAC: Authorizing `identity:get_user(user_id=042afe31291a46d19f7155c3a7b637b7)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:14.372332 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888316cb-2341-4a5e-83ba-e438dfaf90a9 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:14.373918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 562/1127] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:46:14 2022] GET /identity/v3/users/042afe31291a46d19f7155c3a7b637b7 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:14.379680 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13874020-d57c-413b-82a5-1a075b9977bb None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:14.394563 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:14.482584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13874020-d57c-413b-82a5-1a075b9977bb None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:14.517556 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3dea81-fc39-405f-b5fc-6664eabea4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:14.518152 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3dea81-fc39-405f-b5fc-6664eabea4af None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:14.518454 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3dea81-fc39-405f-b5fc-6664eabea4af None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:14.518729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3dea81-fc39-405f-b5fc-6664eabea4af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:14.584900 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3dea81-fc39-405f-b5fc-6664eabea4af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:14.586363 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3dea81-fc39-405f-b5fc-6664eabea4af None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:14.600375 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:14.683518 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 566/1128] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:46:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:46:14.689441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888316cb-2341-4a5e-83ba-e438dfaf90a9 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:14.786140 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:14.789100 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888316cb-2341-4a5e-83ba-e438dfaf90a9 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:14.823979 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd36b7-44d9-451f-bc07-4689d010cae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:14.824612 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd36b7-44d9-451f-bc07-4689d010cae3 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:14.824890 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd36b7-44d9-451f-bc07-4689d010cae3 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:14.825162 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd36b7-44d9-451f-bc07-4689d010cae3 None admin] PATH_INFO: `/v3/projects/85b7a190e444441c8361d1f925e0dcff` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:14.826288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fd36b7-44d9-451f-bc07-4689d010cae3 None admin] RBAC: Authorizing `identity:get_project(project_id=85b7a190e444441c8361d1f925e0dcff)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:14.827765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fd36b7-44d9-451f-bc07-4689d010cae3 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:14.829412 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 563/1129] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:46:14 2022] GET /identity/v3/projects/85b7a190e444441c8361d1f925e0dcff => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:14.835785 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3dea81-fc39-405f-b5fc-6664eabea4af None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:14.934797 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3dea81-fc39-405f-b5fc-6664eabea4af None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:14.947118 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:14.967781 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0810df-ccc6-44bf-9c6e-624dd7aa1fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:14.968379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0810df-ccc6-44bf-9c6e-624dd7aa1fa2 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:14.968652 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0810df-ccc6-44bf-9c6e-624dd7aa1fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:14.968924 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0810df-ccc6-44bf-9c6e-624dd7aa1fa2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:14.969636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0810df-ccc6-44bf-9c6e-624dd7aa1fa2 None admin] RBAC: Adding query filter params (role.id=31d4b1dc082c41f7b6f18c338e7b51df, scope.project.id=85b7a190e444441c8361d1f925e0dcff, user.id=042afe31291a46d19f7155c3a7b637b7) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:46:14.969938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0810df-ccc6-44bf-9c6e-624dd7aa1fa2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:14.971414 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0810df-ccc6-44bf-9c6e-624dd7aa1fa2 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:14.977082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 567/1130] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:46:14 2022] GET /identity/v3/role_assignments?user.id=042afe31291a46d19f7155c3a7b637b7&scope.project.id=85b7a190e444441c8361d1f925e0dcff&role.id=31d4b1dc082c41f7b6f18c338e7b51df => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:16.932575 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16288b7b-0583-496c-ae2f-313057fedceb None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:16.933107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16288b7b-0583-496c-ae2f-313057fedceb None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:16.933107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16288b7b-0583-496c-ae2f-313057fedceb None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:16.934488 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 564/1131] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:46:16.939708 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fc20f-7570-4269-8795-2de6799e9df1 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:16.940117 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fc20f-7570-4269-8795-2de6799e9df1 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:16.940251 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fc20f-7570-4269-8795-2de6799e9df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:16.957834 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-fb8fc20f-7570-4269-8795-2de6799e9df1 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:46:16.959032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-fb8fc20f-7570-4269-8795-2de6799e9df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:46:17.023388 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 568/1132] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:46:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:46:17.292592 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16288b7b-0583-496c-ae2f-313057fedceb None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:17.393425 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16288b7b-0583-496c-ae2f-313057fedceb None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:17.426088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db040eb-2c68-4fde-8a82-138ef6a2b50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:17.426727 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db040eb-2c68-4fde-8a82-138ef6a2b50e None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:17.427000 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db040eb-2c68-4fde-8a82-138ef6a2b50e None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:17.427265 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db040eb-2c68-4fde-8a82-138ef6a2b50e None admin] PATH_INFO: `/v3/domains/default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:17.428358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db040eb-2c68-4fde-8a82-138ef6a2b50e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:17.429766 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db040eb-2c68-4fde-8a82-138ef6a2b50e None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:17.431339 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 565/1133] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:46:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:46:17.437222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8fc20f-7570-4269-8795-2de6799e9df1 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:17.533501 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8fc20f-7570-4269-8795-2de6799e9df1 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:17.540612 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:17.567021 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f12c9-545b-4e70-a9a1-a0849e5cd9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:17.567553 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f12c9-545b-4e70-a9a1-a0849e5cd9da None admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:17.567830 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f12c9-545b-4e70-a9a1-a0849e5cd9da None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:17.568102 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f12c9-545b-4e70-a9a1-a0849e5cd9da None admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:17.568751 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8f12c9-545b-4e70-a9a1-a0849e5cd9da None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:17.570178 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8f12c9-545b-4e70-a9a1-a0849e5cd9da None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:17.663169 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 569/1134] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Aug 31 10:46:17 2022] POST /identity/v3/projects => generated 311 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:46:19.584099 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba9523-fba1-4ace-9a4b-1663a69d7bee None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:19.584635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba9523-fba1-4ace-9a4b-1663a69d7bee None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:19.584635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba9523-fba1-4ace-9a4b-1663a69d7bee None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:19.586046 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 566/1135] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:46:19.591238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83c161-b8e1-4500-b6fc-1dc7cd9dc621 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:19.591685 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83c161-b8e1-4500-b6fc-1dc7cd9dc621 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:19.591777 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83c161-b8e1-4500-b6fc-1dc7cd9dc621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:19.611870 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-1d83c161-b8e1-4500-b6fc-1dc7cd9dc621 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:46:19.613076 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-1d83c161-b8e1-4500-b6fc-1dc7cd9dc621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:46:19.680428 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 570/1136] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:46:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:46:19.950237 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba9523-fba1-4ace-9a4b-1663a69d7bee None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:19.986194 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:20.050863 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba9523-fba1-4ace-9a4b-1663a69d7bee None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:20.083820 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026cf3cc-d3de-491d-916d-f2137b6751cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:20.084448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cf3cc-d3de-491d-916d-f2137b6751cd None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:20.084722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cf3cc-d3de-491d-916d-f2137b6751cd None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:20.084988 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cf3cc-d3de-491d-916d-f2137b6751cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:20.086120 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026cf3cc-d3de-491d-916d-f2137b6751cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:20.088080 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026cf3cc-d3de-491d-916d-f2137b6751cd None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:20.089708 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 567/1137] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:46:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:20.095728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d83c161-b8e1-4500-b6fc-1dc7cd9dc621 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:20.187631 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:20.200236 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d83c161-b8e1-4500-b6fc-1dc7cd9dc621 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:20.234443 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b73fed-e087-4fe0-94ac-29aaa8d4c0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:20.234987 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b73fed-e087-4fe0-94ac-29aaa8d4c0b0 None admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:20.235265 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b73fed-e087-4fe0-94ac-29aaa8d4c0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:20.235536 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b73fed-e087-4fe0-94ac-29aaa8d4c0b0 None admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:20.236194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b73fed-e087-4fe0-94ac-29aaa8d4c0b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:20.237621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b73fed-e087-4fe0-94ac-29aaa8d4c0b0 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:20.343709 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 571/1138] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 31 10:46:20 2022] POST /identity/v3/users => generated 286 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:46:22.354744 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b759a-c163-4fd1-8884-ad3f0310eaef None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:22.355268 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b759a-c163-4fd1-8884-ad3f0310eaef None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:22.355268 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b759a-c163-4fd1-8884-ad3f0310eaef None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:22.356610 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 568/1139] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:46:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:46:22.361732 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c6e89-77b6-4173-bdbb-c052431462a7 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:22.362101 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c6e89-77b6-4173-bdbb-c052431462a7 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:22.362235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c6e89-77b6-4173-bdbb-c052431462a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:22.380580 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-654c6e89-77b6-4173-bdbb-c052431462a7 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:46:22.381766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-654c6e89-77b6-4173-bdbb-c052431462a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:46:22.448225 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 572/1140] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:46:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:46:22.718828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b759a-c163-4fd1-8884-ad3f0310eaef None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:22.818005 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b759a-c163-4fd1-8884-ad3f0310eaef None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:22.820308 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:22.851726 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b5c76-3333-4e2e-b7ed-0f935fd9c3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:22.852591 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b5c76-3333-4e2e-b7ed-0f935fd9c3c5 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:22.853055 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b5c76-3333-4e2e-b7ed-0f935fd9c3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:22.854628 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b5c76-3333-4e2e-b7ed-0f935fd9c3c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:22.864572 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b5c76-3333-4e2e-b7ed-0f935fd9c3c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:46:22.869923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-e49b5c76-3333-4e2e-b7ed-0f935fd9c3c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:46:22.869923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:46:22.869923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:46:22.869923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:46:22.869923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:46:22.869923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:46:22.869923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:46:22.869923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:46:22.869923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:46:22.869923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:46:22.869923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:46:22.869923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:46:22.869923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:46:22.869923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:46:22.869923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:46:22.869923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:46:22.871032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:46:22.871032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:46:22.871032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:46:22.871032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:46:22.871032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:46:22.871032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:46:22.871032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:46:22.871032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:46:22.871032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:46:22.871032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:46:22.871032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:46:22.871032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:46:22.871032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:46:22.871032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:46:22.871032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:46:22.871032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:46:22.872167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:46:22.872167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:46:22.872167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:46:22.872167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:46:22.872167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:46:22.872167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:46:22.872167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:46:22.872167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:46:22.872167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:46:22.872167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:46:22.872167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:46:22.872167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:46:22.872167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 569/1141] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:46:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:46:22.877189 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c6e89-77b6-4173-bdbb-c052431462a7 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:22.950055 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:22.975303 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c6e89-77b6-4173-bdbb-c052431462a7 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:23.012838 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b543c-35d9-4fd5-b9c7-4c88a2c28862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:23.013500 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b543c-35d9-4fd5-b9c7-4c88a2c28862 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:23.014157 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b543c-35d9-4fd5-b9c7-4c88a2c28862 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:23.014454 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b543c-35d9-4fd5-b9c7-4c88a2c28862 None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:23.015140 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b543c-35d9-4fd5-b9c7-4c88a2c28862 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:46:23.015445 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b543c-35d9-4fd5-b9c7-4c88a2c28862 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:23.016828 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b543c-35d9-4fd5-b9c7-4c88a2c28862 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:23.026465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 573/1142] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:46:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:23.032285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b5c76-3333-4e2e-b7ed-0f935fd9c3c5 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:23.136344 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b5c76-3333-4e2e-b7ed-0f935fd9c3c5 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:23.170963 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058b8b9-904f-42aa-b853-ad0a81441713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:23.171605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058b8b9-904f-42aa-b853-ad0a81441713 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:23.171881 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058b8b9-904f-42aa-b853-ad0a81441713 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:23.172152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058b8b9-904f-42aa-b853-ad0a81441713 None admin] PATH_INFO: `/v3/users/641154741ff346fe842c719db2f08a8d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:23.185089 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5058b8b9-904f-42aa-b853-ad0a81441713 None admin] RBAC: Authorizing `identity:get_user(user_id=641154741ff346fe842c719db2f08a8d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:23.186557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5058b8b9-904f-42aa-b853-ad0a81441713 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:23.188149 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 570/1143] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:46:23 2022] GET /identity/v3/users/641154741ff346fe842c719db2f08a8d => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:23.194074 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b543c-35d9-4fd5-b9c7-4c88a2c28862 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:23.291058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b543c-35d9-4fd5-b9c7-4c88a2c28862 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:23.323125 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e26b22-00fa-406e-8c58-40223829acb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:23.323719 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e26b22-00fa-406e-8c58-40223829acb3 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:23.323996 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e26b22-00fa-406e-8c58-40223829acb3 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:23.324265 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e26b22-00fa-406e-8c58-40223829acb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:23.327541 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:23.388945 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e26b22-00fa-406e-8c58-40223829acb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:23.390375 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e26b22-00fa-406e-8c58-40223829acb3 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:23.485301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 574/1144] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:46:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:46:23.491295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058b8b9-904f-42aa-b853-ad0a81441713 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:23.515880 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:23.588575 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058b8b9-904f-42aa-b853-ad0a81441713 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:23.621679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cadc4a9-9800-4218-8a9b-0c50e1b0e92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:23.622297 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cadc4a9-9800-4218-8a9b-0c50e1b0e92f None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:23.622603 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cadc4a9-9800-4218-8a9b-0c50e1b0e92f None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:23.622878 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cadc4a9-9800-4218-8a9b-0c50e1b0e92f None admin] PATH_INFO: `/v3/projects/7c005ad37b684908bc7a59de64f792cb` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:23.623996 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cadc4a9-9800-4218-8a9b-0c50e1b0e92f None admin] RBAC: Authorizing `identity:get_project(project_id=7c005ad37b684908bc7a59de64f792cb)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:23.625413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cadc4a9-9800-4218-8a9b-0c50e1b0e92f None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:23.627068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 571/1145] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:46:23 2022] GET /identity/v3/projects/7c005ad37b684908bc7a59de64f792cb => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:23.633177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e26b22-00fa-406e-8c58-40223829acb3 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:23.667814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:23.728929 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e26b22-00fa-406e-8c58-40223829acb3 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:23.761324 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f767e02d-77e5-414d-8b5e-3abf8fed3f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:23.761905 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767e02d-77e5-414d-8b5e-3abf8fed3f56 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:23.762176 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767e02d-77e5-414d-8b5e-3abf8fed3f56 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:23.762473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767e02d-77e5-414d-8b5e-3abf8fed3f56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:23.763182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f767e02d-77e5-414d-8b5e-3abf8fed3f56 None admin] RBAC: Adding query filter params (role.id=13ca33d694c645d6920930d39dc52a0d, scope.project.id=7c005ad37b684908bc7a59de64f792cb, user.id=641154741ff346fe842c719db2f08a8d) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:46:23.763483 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f767e02d-77e5-414d-8b5e-3abf8fed3f56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:23.764883 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f767e02d-77e5-414d-8b5e-3abf8fed3f56 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:23.770002 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 575/1146] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:46:23 2022] GET /identity/v3/role_assignments?user.id=641154741ff346fe842c719db2f08a8d&scope.project.id=7c005ad37b684908bc7a59de64f792cb&role.id=13ca33d694c645d6920930d39dc52a0d => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:25.668770 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2702e4-34cc-40d8-a47c-740ccb2d87c4 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:25.669326 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2702e4-34cc-40d8-a47c-740ccb2d87c4 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:25.669326 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2702e4-34cc-40d8-a47c-740ccb2d87c4 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:25.670689 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 572/1147] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:46:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:46:25.676084 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669192a2-8bb6-46a8-b332-353e61af40a2 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:25.676497 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669192a2-8bb6-46a8-b332-353e61af40a2 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:25.676606 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669192a2-8bb6-46a8-b332-353e61af40a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:25.694154 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-669192a2-8bb6-46a8-b332-353e61af40a2 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:46:25.695358 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-669192a2-8bb6-46a8-b332-353e61af40a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:46:25.760728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 576/1148] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:46:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:46:26.026947 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2702e4-34cc-40d8-a47c-740ccb2d87c4 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:26.117765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:26.128682 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2702e4-34cc-40d8-a47c-740ccb2d87c4 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:26.164289 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ec4e3-66b0-4cb0-9a77-e2c1bc8f330a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:26.164914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ec4e3-66b0-4cb0-9a77-e2c1bc8f330a None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:26.165185 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ec4e3-66b0-4cb0-9a77-e2c1bc8f330a None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:26.165447 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ec4e3-66b0-4cb0-9a77-e2c1bc8f330a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:26.176173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ec4e3-66b0-4cb0-9a77-e2c1bc8f330a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:46:26.180856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-db9ec4e3-66b0-4cb0-9a77-e2c1bc8f330a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:46:26.180856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:46:26.180856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:46:26.180856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:46:26.180856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:46:26.180856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:46:26.180856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:46:26.180856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:46:26.180856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:46:26.180856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:46:26.180856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:46:26.180856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:46:26.180856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:46:26.180856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:46:26.180856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:46:26.180856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:46:26.182088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:46:26.182088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:46:26.182088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:46:26.182088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:46:26.182088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:46:26.182088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:46:26.182088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:46:26.182088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:46:26.182088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:46:26.182088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:46:26.182088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:46:26.182088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:46:26.182088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:46:26.182088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:46:26.182088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:46:26.182088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:46:26.183240 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:46:26.183240 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:46:26.183240 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:46:26.183240 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:46:26.183240 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:46:26.183240 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:46:26.183240 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:46:26.183240 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:46:26.183240 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:46:26.183240 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:46:26.183240 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:46:26.183240 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:46:26.183240 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 573/1149] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:46:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:46:26.188086 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669192a2-8bb6-46a8-b332-353e61af40a2 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:26.217822 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:26.285845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669192a2-8bb6-46a8-b332-353e61af40a2 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:26.318351 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae83ae-8d21-4715-88ae-ada30ed91c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:26.318982 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae83ae-8d21-4715-88ae-ada30ed91c19 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:26.319259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae83ae-8d21-4715-88ae-ada30ed91c19 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:26.319530 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae83ae-8d21-4715-88ae-ada30ed91c19 None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:26.320180 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbae83ae-8d21-4715-88ae-ada30ed91c19 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:46:26.320480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbae83ae-8d21-4715-88ae-ada30ed91c19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:26.322450 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbae83ae-8d21-4715-88ae-ada30ed91c19 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:26.330427 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 577/1150] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:46:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:26.336331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ec4e3-66b0-4cb0-9a77-e2c1bc8f330a None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:26.434708 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ec4e3-66b0-4cb0-9a77-e2c1bc8f330a None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:26.468101 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5cece-79a2-42dd-8847-de4b74c59ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:26.468725 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5cece-79a2-42dd-8847-de4b74c59ea6 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:26.469002 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5cece-79a2-42dd-8847-de4b74c59ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:26.469269 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5cece-79a2-42dd-8847-de4b74c59ea6 None admin] PATH_INFO: `/v3/users/641154741ff346fe842c719db2f08a8d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:26.470373 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd5cece-79a2-42dd-8847-de4b74c59ea6 None admin] RBAC: Authorizing `identity:get_user(user_id=641154741ff346fe842c719db2f08a8d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:26.471817 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd5cece-79a2-42dd-8847-de4b74c59ea6 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:26.473392 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 574/1151] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:46:26 2022] GET /identity/v3/users/641154741ff346fe842c719db2f08a8d => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:26.479476 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae83ae-8d21-4715-88ae-ada30ed91c19 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:26.571687 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:26.581355 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae83ae-8d21-4715-88ae-ada30ed91c19 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:26.615226 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30719a-721e-4720-ae49-168c3b81b03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:26.615843 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30719a-721e-4720-ae49-168c3b81b03f None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:26.616120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30719a-721e-4720-ae49-168c3b81b03f None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:26.616390 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30719a-721e-4720-ae49-168c3b81b03f None admin] PATH_INFO: `/v3/projects/7c005ad37b684908bc7a59de64f792cb` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:26.617508 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b30719a-721e-4720-ae49-168c3b81b03f None admin] RBAC: Authorizing `identity:get_project(project_id=7c005ad37b684908bc7a59de64f792cb)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:26.620047 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b30719a-721e-4720-ae49-168c3b81b03f None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:26.621706 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 578/1152] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:46:26 2022] GET /identity/v3/projects/7c005ad37b684908bc7a59de64f792cb => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:46:26.627987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5cece-79a2-42dd-8847-de4b74c59ea6 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:26.647848 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:26.727088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5cece-79a2-42dd-8847-de4b74c59ea6 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:26.761126 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede3f84-f206-4e74-ac30-bc9b30bcb228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:26.761763 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede3f84-f206-4e74-ac30-bc9b30bcb228 None admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:26.762035 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede3f84-f206-4e74-ac30-bc9b30bcb228 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:26.762304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede3f84-f206-4e74-ac30-bc9b30bcb228 None admin] PATH_INFO: `/v3/projects/7c005ad37b684908bc7a59de64f792cb/users/641154741ff346fe842c719db2f08a8d/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:26.764209 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ede3f84-f206-4e74-ac30-bc9b30bcb228 None admin] RBAC: Authorizing `identity:create_grant(project_id=7c005ad37b684908bc7a59de64f792cb, user_id=641154741ff346fe842c719db2f08a8d, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:26.765629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ede3f84-f206-4e74-ac30-bc9b30bcb228 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:26.865002 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 575/1153] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 31 10:46:26 2022] PUT /identity/v3/projects/7c005ad37b684908bc7a59de64f792cb/users/641154741ff346fe842c719db2f08a8d/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:46:28.761920 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3d8fd-e866-4654-86c6-898ac05443b4 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:28.762473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3d8fd-e866-4654-86c6-898ac05443b4 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:28.762567 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3d8fd-e866-4654-86c6-898ac05443b4 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:28.763846 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 579/1154] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:46:28.769065 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150a790-3928-43bb-9c6e-d597e93d6da2 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:28.769445 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150a790-3928-43bb-9c6e-d597e93d6da2 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:28.769617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150a790-3928-43bb-9c6e-d597e93d6da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:28.788033 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-f150a790-3928-43bb-9c6e-d597e93d6da2 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:46:28.789239 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-f150a790-3928-43bb-9c6e-d597e93d6da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:46:28.856806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 576/1155] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:46:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:46:29.131569 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb3d8fd-e866-4654-86c6-898ac05443b4 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:29.232881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb3d8fd-e866-4654-86c6-898ac05443b4 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:29.267900 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d37ecd-f86b-4662-a9b4-de132ea205f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:29.274668 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d37ecd-f86b-4662-a9b4-de132ea205f0 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:29.274668 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d37ecd-f86b-4662-a9b4-de132ea205f0 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:29.274668 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d37ecd-f86b-4662-a9b4-de132ea205f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:29.281089 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d37ecd-f86b-4662-a9b4-de132ea205f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:46:29.285932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-a6d37ecd-f86b-4662-a9b4-de132ea205f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:46:29.285932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:46:29.285932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:46:29.285932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:46:29.285932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:46:29.285932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:46:29.285932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:46:29.285932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:46:29.285932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:46:29.285932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:46:29.285932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:46:29.285932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:46:29.285932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:46:29.285932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:46:29.285932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:46:29.285932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:46:29.287217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:46:29.287217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:46:29.287217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:46:29.287217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:46:29.287217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:46:29.287217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:46:29.287217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:46:29.287217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:46:29.287217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:46:29.287217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:46:29.287217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:46:29.287217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:46:29.287217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:46:29.287217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:46:29.287217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:46:29.287217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:46:29.288242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:46:29.288242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:46:29.288242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:46:29.288242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:46:29.288242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:46:29.288242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:46:29.288242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:46:29.288242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:46:29.288242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:46:29.288242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:46:29.288242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:46:29.288242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:46:29.288242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 580/1156] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:46:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:46:29.293759 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150a790-3928-43bb-9c6e-d597e93d6da2 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:29.304297 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:29.393380 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150a790-3928-43bb-9c6e-d597e93d6da2 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:29.426287 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d255b-b816-430e-9c95-bc2ac942ccde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:29.426911 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d255b-b816-430e-9c95-bc2ac942ccde None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:29.427184 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d255b-b816-430e-9c95-bc2ac942ccde None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:29.427450 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d255b-b816-430e-9c95-bc2ac942ccde None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:29.428100 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d255b-b816-430e-9c95-bc2ac942ccde None admin] RBAC: Adding query filter params (name=admin) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:46:29.428423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d255b-b816-430e-9c95-bc2ac942ccde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:29.429775 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d255b-b816-430e-9c95-bc2ac942ccde None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:29.438149 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 577/1157] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:46:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:29.443966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d37ecd-f86b-4662-a9b4-de132ea205f0 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:29.466676 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:29.541239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d37ecd-f86b-4662-a9b4-de132ea205f0 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:29.574029 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528a833-b3ba-4cd5-938a-dca034f56cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:29.574676 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528a833-b3ba-4cd5-938a-dca034f56cdc None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:29.574946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528a833-b3ba-4cd5-938a-dca034f56cdc None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:29.575211 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528a833-b3ba-4cd5-938a-dca034f56cdc None admin] PATH_INFO: `/v3/users/641154741ff346fe842c719db2f08a8d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:29.576310 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528a833-b3ba-4cd5-938a-dca034f56cdc None admin] RBAC: Authorizing `identity:get_user(user_id=641154741ff346fe842c719db2f08a8d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:29.577723 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528a833-b3ba-4cd5-938a-dca034f56cdc None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:29.579300 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 581/1158] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:46:29 2022] GET /identity/v3/users/641154741ff346fe842c719db2f08a8d => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:29.585226 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d255b-b816-430e-9c95-bc2ac942ccde None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:29.653303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:29.683159 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d255b-b816-430e-9c95-bc2ac942ccde None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:29.716712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18914c-9585-4203-9c32-0ded287be747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:29.717305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18914c-9585-4203-9c32-0ded287be747 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:29.717580 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18914c-9585-4203-9c32-0ded287be747 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:29.717851 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18914c-9585-4203-9c32-0ded287be747 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:29.783669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18914c-9585-4203-9c32-0ded287be747 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:29.785129 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18914c-9585-4203-9c32-0ded287be747 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:29.850241 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:29.887389 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 578/1159] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:46:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:46:29.893269 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528a833-b3ba-4cd5-938a-dca034f56cdc None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:29.980331 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:29.991617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528a833-b3ba-4cd5-938a-dca034f56cdc None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:30.025383 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e2458-fe87-45c2-9fad-3c1dc4015dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:30.026007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e2458-fe87-45c2-9fad-3c1dc4015dd0 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:30.026282 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e2458-fe87-45c2-9fad-3c1dc4015dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:30.026582 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e2458-fe87-45c2-9fad-3c1dc4015dd0 None admin] PATH_INFO: `/v3/projects/7c005ad37b684908bc7a59de64f792cb` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:30.027696 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e2458-fe87-45c2-9fad-3c1dc4015dd0 None admin] RBAC: Authorizing `identity:get_project(project_id=7c005ad37b684908bc7a59de64f792cb)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:30.029139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e2458-fe87-45c2-9fad-3c1dc4015dd0 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:30.030793 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 582/1160] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:46:29 2022] GET /identity/v3/projects/7c005ad37b684908bc7a59de64f792cb => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:30.036860 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18914c-9585-4203-9c32-0ded287be747 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:30.133694 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18914c-9585-4203-9c32-0ded287be747 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:30.166909 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1814a3a-e12e-4e86-a52d-a27c037722ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:30.167500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1814a3a-e12e-4e86-a52d-a27c037722ce None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:30.167775 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1814a3a-e12e-4e86-a52d-a27c037722ce None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:30.168045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1814a3a-e12e-4e86-a52d-a27c037722ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:30.168758 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1814a3a-e12e-4e86-a52d-a27c037722ce None admin] RBAC: Adding query filter params (role.id=13ca33d694c645d6920930d39dc52a0d, scope.project.id=7c005ad37b684908bc7a59de64f792cb, user.id=641154741ff346fe842c719db2f08a8d) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:46:30.169061 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1814a3a-e12e-4e86-a52d-a27c037722ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:30.170496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1814a3a-e12e-4e86-a52d-a27c037722ce None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:30.176041 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 579/1161] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:46:30 2022] GET /identity/v3/role_assignments?user.id=641154741ff346fe842c719db2f08a8d&scope.project.id=7c005ad37b684908bc7a59de64f792cb&role.id=13ca33d694c645d6920930d39dc52a0d => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:32.145768 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e530e8-a150-4f7e-9579-f3ecbed07d27 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:32.146293 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e530e8-a150-4f7e-9579-f3ecbed07d27 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:32.146293 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e530e8-a150-4f7e-9579-f3ecbed07d27 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:32.147668 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 583/1162] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:46:32.154196 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45ae3c-735e-435d-a8b8-fe072bc24310 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:32.154641 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45ae3c-735e-435d-a8b8-fe072bc24310 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:32.154796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45ae3c-735e-435d-a8b8-fe072bc24310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:32.173228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-5a45ae3c-735e-435d-a8b8-fe072bc24310 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:46:32.174450 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-5a45ae3c-735e-435d-a8b8-fe072bc24310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:46:32.201318 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:32.241547 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 580/1163] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:46:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:46:32.506449 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e530e8-a150-4f7e-9579-f3ecbed07d27 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:32.604209 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e530e8-a150-4f7e-9579-f3ecbed07d27 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:32.637148 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9cd3af-9e8f-4976-ad31-2480fef36519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:32.637737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cd3af-9e8f-4976-ad31-2480fef36519 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:32.638011 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cd3af-9e8f-4976-ad31-2480fef36519 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:32.638278 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cd3af-9e8f-4976-ad31-2480fef36519 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:32.661528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:32.704187 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9cd3af-9e8f-4976-ad31-2480fef36519 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:32.705637 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9cd3af-9e8f-4976-ad31-2480fef36519 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:32.803744 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 584/1164] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:46:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:46:32.810966 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45ae3c-735e-435d-a8b8-fe072bc24310 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:32.908168 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45ae3c-735e-435d-a8b8-fe072bc24310 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:32.942314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d8ac9-5f46-4ac9-99f0-198d8979078b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:32.942973 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d8ac9-5f46-4ac9-99f0-198d8979078b None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:32.943249 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d8ac9-5f46-4ac9-99f0-198d8979078b None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:32.943514 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d8ac9-5f46-4ac9-99f0-198d8979078b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:32.950210 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d8ac9-5f46-4ac9-99f0-198d8979078b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:32.951668 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d8ac9-5f46-4ac9-99f0-198d8979078b None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:32.962891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-165d8ac9-5f46-4ac9-99f0-198d8979078b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 10:46:32.962891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:46:32.962891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 31 10:46:32.962891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 31 10:46:32.962891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:46:32.962891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:46:32.962891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:46:32.962891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:46:32.962891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 31 10:46:32.962891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:46:32.962891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:46:32.962891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:46:32.962891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:46:32.962891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:46:32.962891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:46:32.962891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:46:32.962891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:46:32.964139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:46:32.964139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:46:32.964139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:46:32.964139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:46:32.964139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:46:32.964139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:46:32.964139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:46:32.964139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:46:32.964139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:46:32.964139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:46:32.964139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:46:32.964139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:46:32.964139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:46:32.964139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:46:32.964139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:46:32.964139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:46:32.965250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:46:32.965250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:46:32.965250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:46:32.965250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:46:32.965250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:46:32.965250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:46:32.965250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:46:32.965250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:46:32.965250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:46:32.965250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:46:32.965250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 31 10:46:32.965250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:46:32.965250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 10:46:32.965250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:46:32.965250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 581/1165] 10.222.0.16 () {56 vars in 1117 bytes} [Wed Aug 31 10:46:32 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:46:32.970082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9cd3af-9e8f-4976-ad31-2480fef36519 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:33.025664 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:33.069183 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9cd3af-9e8f-4976-ad31-2480fef36519 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:33.101212 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3591f9-99a9-40af-ae79-e9ec5c180d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:33.101816 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3591f9-99a9-40af-ae79-e9ec5c180d20 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:33.102091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3591f9-99a9-40af-ae79-e9ec5c180d20 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:33.102409 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3591f9-99a9-40af-ae79-e9ec5c180d20 None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:33.103644 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3591f9-99a9-40af-ae79-e9ec5c180d20 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:46:33.103944 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3591f9-99a9-40af-ae79-e9ec5c180d20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:33.105313 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3591f9-99a9-40af-ae79-e9ec5c180d20 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:33.112569 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 585/1166] 10.222.0.16 () {56 vars in 1115 bytes} [Wed Aug 31 10:46:32 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:35.043523 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93090357-13e7-47cf-a8af-0e3bf6e520e9 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:35.044098 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93090357-13e7-47cf-a8af-0e3bf6e520e9 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:35.044098 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93090357-13e7-47cf-a8af-0e3bf6e520e9 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:35.045415 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 582/1167] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:46:35.051433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f4c4c-bb2a-4d64-b960-9ec7181cefda None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:35.051910 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f4c4c-bb2a-4d64-b960-9ec7181cefda None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:35.052039 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f4c4c-bb2a-4d64-b960-9ec7181cefda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:35.070898 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-314f4c4c-bb2a-4d64-b960-9ec7181cefda None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:46:35.072305 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-314f4c4c-bb2a-4d64-b960-9ec7181cefda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:46:35.141918 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 586/1168] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:46:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:46:35.413659 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93090357-13e7-47cf-a8af-0e3bf6e520e9 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:35.445685 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:35.554692 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93090357-13e7-47cf-a8af-0e3bf6e520e9 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:35.601421 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fa7c7-8b1e-45d9-9456-160ba79e19f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:35.602048 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fa7c7-8b1e-45d9-9456-160ba79e19f8 None admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:35.602320 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fa7c7-8b1e-45d9-9456-160ba79e19f8 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:35.602612 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fa7c7-8b1e-45d9-9456-160ba79e19f8 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:35.603934 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8fa7c7-8b1e-45d9-9456-160ba79e19f8 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:35.605306 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8fa7c7-8b1e-45d9-9456-160ba79e19f8 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:35.686924 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 583/1169] 10.222.0.16 () {60 vars in 1138 bytes} [Wed Aug 31 10:46:35 2022] POST /identity/v3/domains => generated 267 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:46:37.619077 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db145245-3206-4dab-91de-822b39ef9c46 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:37.619635 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db145245-3206-4dab-91de-822b39ef9c46 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:37.619635 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db145245-3206-4dab-91de-822b39ef9c46 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:37.620930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 587/1170] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:46:37.626025 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00427e-4f8a-4b79-a47a-54a908f52950 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:37.626473 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00427e-4f8a-4b79-a47a-54a908f52950 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:37.626586 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00427e-4f8a-4b79-a47a-54a908f52950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:37.644791 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-5d00427e-4f8a-4b79-a47a-54a908f52950 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:46:37.645973 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-5d00427e-4f8a-4b79-a47a-54a908f52950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:46:37.710919 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 584/1171] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:46:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:46:37.977296 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db145245-3206-4dab-91de-822b39ef9c46 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:38.026964 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:38.075566 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db145245-3206-4dab-91de-822b39ef9c46 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:38.108264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25374e40-c2c0-45d0-b27a-b66c1fc648b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:38.108893 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25374e40-c2c0-45d0-b27a-b66c1fc648b8 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:38.109170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25374e40-c2c0-45d0-b27a-b66c1fc648b8 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:38.109441 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25374e40-c2c0-45d0-b27a-b66c1fc648b8 None admin] PATH_INFO: `/v3/domains/cc45324d377248b48ef89ea0dc995822` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:38.121393 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25374e40-c2c0-45d0-b27a-b66c1fc648b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=cc45324d377248b48ef89ea0dc995822)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:38.122867 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25374e40-c2c0-45d0-b27a-b66c1fc648b8 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:38.124436 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 588/1172] 10.222.0.16 () {56 vars in 1183 bytes} [Wed Aug 31 10:46:37 2022] GET /identity/v3/domains/cc45324d377248b48ef89ea0dc995822 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:38.130130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00427e-4f8a-4b79-a47a-54a908f52950 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:38.145271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:38.229343 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00427e-4f8a-4b79-a47a-54a908f52950 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:38.263018 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50bd911-8dbe-4cd9-9f4e-62250e447d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:38.263616 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50bd911-8dbe-4cd9-9f4e-62250e447d0f None admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:38.263890 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50bd911-8dbe-4cd9-9f4e-62250e447d0f None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:38.264161 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50bd911-8dbe-4cd9-9f4e-62250e447d0f None admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:38.264813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50bd911-8dbe-4cd9-9f4e-62250e447d0f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:38.266227 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50bd911-8dbe-4cd9-9f4e-62250e447d0f None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:38.441848 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 585/1173] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Aug 31 10:46:38 2022] POST /identity/v3/projects => generated 361 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:46:40.350820 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97debf9-8d35-439e-bd00-e155752ca3d9 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:40.351355 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97debf9-8d35-439e-bd00-e155752ca3d9 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:40.351355 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97debf9-8d35-439e-bd00-e155752ca3d9 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:40.353470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 589/1174] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:46:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:46:40.358616 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac538194-4e11-48b9-a314-eff108eaab23 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:40.359034 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac538194-4e11-48b9-a314-eff108eaab23 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:40.359130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac538194-4e11-48b9-a314-eff108eaab23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:40.377226 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-ac538194-4e11-48b9-a314-eff108eaab23 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:46:40.378441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-ac538194-4e11-48b9-a314-eff108eaab23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:46:40.435918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:40.443086 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 586/1175] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:46:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:46:40.711209 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97debf9-8d35-439e-bd00-e155752ca3d9 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:40.809474 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97debf9-8d35-439e-bd00-e155752ca3d9 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:40.819719 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:40.842278 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d32c9-ba4b-4f28-8f6f-5d2b9d8eedce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:40.842920 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d32c9-ba4b-4f28-8f6f-5d2b9d8eedce None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:40.843197 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d32c9-ba4b-4f28-8f6f-5d2b9d8eedce None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:40.843467 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d32c9-ba4b-4f28-8f6f-5d2b9d8eedce None admin] PATH_INFO: `/v3/domains/cc45324d377248b48ef89ea0dc995822` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:40.844565 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d32c9-ba4b-4f28-8f6f-5d2b9d8eedce None admin] RBAC: Authorizing `identity:get_domain(domain_id=cc45324d377248b48ef89ea0dc995822)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:40.846033 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d32c9-ba4b-4f28-8f6f-5d2b9d8eedce None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:40.847610 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 590/1176] 10.222.0.16 () {56 vars in 1183 bytes} [Wed Aug 31 10:46:40 2022] GET /identity/v3/domains/cc45324d377248b48ef89ea0dc995822 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:40.853284 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac538194-4e11-48b9-a314-eff108eaab23 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:40.950839 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac538194-4e11-48b9-a314-eff108eaab23 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:40.983571 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256db5cd-aaf7-4a22-a458-465bcda002b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:40.984111 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256db5cd-aaf7-4a22-a458-465bcda002b2 None admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:40.984390 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256db5cd-aaf7-4a22-a458-465bcda002b2 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:40.984664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256db5cd-aaf7-4a22-a458-465bcda002b2 None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:40.985306 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256db5cd-aaf7-4a22-a458-465bcda002b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:40.986752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256db5cd-aaf7-4a22-a458-465bcda002b2 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:41.080526 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 587/1177] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 31 10:46:40 2022] POST /identity/v3/users => generated 311 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:46:43.046721 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9dfe63-440e-4227-836c-1075580f0208 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:43.047292 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9dfe63-440e-4227-836c-1075580f0208 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:43.047292 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9dfe63-440e-4227-836c-1075580f0208 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:43.048574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 591/1178] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:46:43.053632 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd98d08-98da-4f1a-8d75-ab678a532a86 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:43.054044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd98d08-98da-4f1a-8d75-ab678a532a86 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:43.054176 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd98d08-98da-4f1a-8d75-ab678a532a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:43.071839 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-1cd98d08-98da-4f1a-8d75-ab678a532a86 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:46:43.073007 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-1cd98d08-98da-4f1a-8d75-ab678a532a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:46:43.120157 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:43.139141 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 588/1179] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:46:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:46:43.409838 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9dfe63-440e-4227-836c-1075580f0208 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:43.507705 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9dfe63-440e-4227-836c-1075580f0208 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:43.541516 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129de55-ce38-4b1e-bc0c-aac38a33b55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:43.542131 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129de55-ce38-4b1e-bc0c-aac38a33b55f None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:43.542432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129de55-ce38-4b1e-bc0c-aac38a33b55f None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:43.542708 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129de55-ce38-4b1e-bc0c-aac38a33b55f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:43.552420 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129de55-ce38-4b1e-bc0c-aac38a33b55f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:46:43.557277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-e129de55-ce38-4b1e-bc0c-aac38a33b55f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:46:43.557277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:46:43.557277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:46:43.557277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:46:43.557277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:46:43.557277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:46:43.557277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:46:43.557277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:46:43.557277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:46:43.557277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:46:43.557277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:46:43.557277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:46:43.557277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:46:43.557277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:46:43.557277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:46:43.557277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:46:43.558468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:46:43.558468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:46:43.558468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:46:43.558468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:46:43.558468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:46:43.558468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:46:43.558468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:46:43.558468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:46:43.558468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:46:43.558468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:46:43.558468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:46:43.558468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:46:43.558468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:46:43.558468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:46:43.558468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:46:43.558468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:46:43.559353 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:46:43.559353 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:46:43.559353 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:46:43.559353 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:46:43.559353 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:46:43.559353 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:46:43.559353 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:46:43.559353 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:46:43.559353 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:46:43.559353 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:46:43.559353 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:46:43.559353 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:46:43.559353 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 592/1180] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:46:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:46:43.564395 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd98d08-98da-4f1a-8d75-ab678a532a86 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:43.659542 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd98d08-98da-4f1a-8d75-ab678a532a86 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:43.692110 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce143c3-a7b9-49ae-b73b-3ac26a6a4735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:43.692698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce143c3-a7b9-49ae-b73b-3ac26a6a4735 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:43.692970 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce143c3-a7b9-49ae-b73b-3ac26a6a4735 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:43.693236 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce143c3-a7b9-49ae-b73b-3ac26a6a4735 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:43.693889 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce143c3-a7b9-49ae-b73b-3ac26a6a4735 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:46:43.694183 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce143c3-a7b9-49ae-b73b-3ac26a6a4735 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:43.695570 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce143c3-a7b9-49ae-b73b-3ac26a6a4735 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:43.703765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 589/1181] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:46:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:43.709465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129de55-ce38-4b1e-bc0c-aac38a33b55f None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:43.750129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:43.807378 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129de55-ce38-4b1e-bc0c-aac38a33b55f None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:43.840473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6195d03-d608-43f4-b247-43e5797e8870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:43.841091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6195d03-d608-43f4-b247-43e5797e8870 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:43.841366 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6195d03-d608-43f4-b247-43e5797e8870 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:43.841633 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6195d03-d608-43f4-b247-43e5797e8870 None admin] PATH_INFO: `/v3/users/ffc09add363e44fca0d12b3466898bce` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:43.854222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6195d03-d608-43f4-b247-43e5797e8870 None admin] RBAC: Authorizing `identity:get_user(user_id=ffc09add363e44fca0d12b3466898bce)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:43.855689 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6195d03-d608-43f4-b247-43e5797e8870 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:43.857263 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 593/1182] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:46:43 2022] GET /identity/v3/users/ffc09add363e44fca0d12b3466898bce => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:43.863297 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce143c3-a7b9-49ae-b73b-3ac26a6a4735 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:43.908374 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:43.961322 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce143c3-a7b9-49ae-b73b-3ac26a6a4735 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:43.993957 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9652e8-93cf-4612-92b7-903e6c3686c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:43.994608 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9652e8-93cf-4612-92b7-903e6c3686c4 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:43.994870 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9652e8-93cf-4612-92b7-903e6c3686c4 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:43.995131 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9652e8-93cf-4612-92b7-903e6c3686c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:44.074250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9652e8-93cf-4612-92b7-903e6c3686c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:44.075718 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9652e8-93cf-4612-92b7-903e6c3686c4 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:44.120815 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:44.177694 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 590/1183] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:46:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 10:46:44.184813 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6195d03-d608-43f4-b247-43e5797e8870 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:44.280132 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:44.285855 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6195d03-d608-43f4-b247-43e5797e8870 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:44.319556 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb5c57-570a-470a-bb7f-e9c0f3f9ed8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:44.320167 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb5c57-570a-470a-bb7f-e9c0f3f9ed8c None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:44.320442 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb5c57-570a-470a-bb7f-e9c0f3f9ed8c None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:44.320709 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb5c57-570a-470a-bb7f-e9c0f3f9ed8c None admin] PATH_INFO: `/v3/projects/1dbfe238dadf49d9a8303aee60ea5797` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:44.322079 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb5c57-570a-470a-bb7f-e9c0f3f9ed8c None admin] RBAC: Authorizing `identity:get_project(project_id=1dbfe238dadf49d9a8303aee60ea5797)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:44.323530 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb5c57-570a-470a-bb7f-e9c0f3f9ed8c None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:44.325122 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 594/1184] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:46:44 2022] GET /identity/v3/projects/1dbfe238dadf49d9a8303aee60ea5797 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:44.331311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9652e8-93cf-4612-92b7-903e6c3686c4 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:44.428104 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9652e8-93cf-4612-92b7-903e6c3686c4 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:44.460994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddacfb5-5d01-44b8-87f1-45469ae4c8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:44.461577 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddacfb5-5d01-44b8-87f1-45469ae4c8b5 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:44.461849 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddacfb5-5d01-44b8-87f1-45469ae4c8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:44.462758 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddacfb5-5d01-44b8-87f1-45469ae4c8b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:44.463547 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddacfb5-5d01-44b8-87f1-45469ae4c8b5 None admin] RBAC: Adding query filter params (role.id=13ca33d694c645d6920930d39dc52a0d, scope.project.id=1dbfe238dadf49d9a8303aee60ea5797, user.id=ffc09add363e44fca0d12b3466898bce) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:46:44.463848 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddacfb5-5d01-44b8-87f1-45469ae4c8b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:44.465238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddacfb5-5d01-44b8-87f1-45469ae4c8b5 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:44.470369 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 591/1185] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:46:44 2022] GET /identity/v3/role_assignments?user.id=ffc09add363e44fca0d12b3466898bce&scope.project.id=1dbfe238dadf49d9a8303aee60ea5797&role.id=13ca33d694c645d6920930d39dc52a0d => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:46.336344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a4161-94fb-478e-a7f3-adca4573d82d None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:46.336883 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a4161-94fb-478e-a7f3-adca4573d82d None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:46.336883 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a4161-94fb-478e-a7f3-adca4573d82d None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:46.338433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 595/1186] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:46:46.343536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9bc32-be69-40fa-860d-fe01bf02fa05 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:46.365386 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9bc32-be69-40fa-860d-fe01bf02fa05 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:46.365386 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9bc32-be69-40fa-860d-fe01bf02fa05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:46.365386 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-c5d9bc32-be69-40fa-860d-fe01bf02fa05 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:46:46.365386 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-c5d9bc32-be69-40fa-860d-fe01bf02fa05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:46:46.366099 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:46.433000 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 592/1187] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:46:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:46:46.699418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a4161-94fb-478e-a7f3-adca4573d82d None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:46.798347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a4161-94fb-478e-a7f3-adca4573d82d None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:46.830756 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258fa35e-2f88-484b-90f5-c8f5c19af7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:46.831367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258fa35e-2f88-484b-90f5-c8f5c19af7f5 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:46.831647 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258fa35e-2f88-484b-90f5-c8f5c19af7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:46.831923 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258fa35e-2f88-484b-90f5-c8f5c19af7f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:46.842278 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258fa35e-2f88-484b-90f5-c8f5c19af7f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:46:46.847031 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-258fa35e-2f88-484b-90f5-c8f5c19af7f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:46:46.847031 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:46:46.847031 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:46:46.847031 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:46:46.847031 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:46:46.847031 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:46:46.847031 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:46:46.847031 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:46:46.847031 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:46:46.847031 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:46:46.847031 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:46:46.847031 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:46:46.847031 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:46:46.847031 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:46:46.847031 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:46:46.847031 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:46:46.848137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:46:46.848137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:46:46.848137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:46:46.848137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:46:46.848137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:46:46.848137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:46:46.848137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:46:46.848137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:46:46.848137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:46:46.848137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:46:46.848137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:46:46.848137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:46:46.848137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:46:46.848137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:46:46.848137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:46:46.848137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:46:46.849012 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:46:46.849012 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:46:46.849012 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:46:46.849012 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:46:46.849012 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:46:46.849012 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:46:46.849012 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:46:46.849012 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:46:46.849012 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:46:46.849012 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:46:46.849012 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:46:46.849012 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:46:46.849012 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 596/1188] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:46:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:46:46.854919 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9bc32-be69-40fa-860d-fe01bf02fa05 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:46.949396 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9bc32-be69-40fa-860d-fe01bf02fa05 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:46.982366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af7b4a-1cae-46d3-b27c-6afbb2d3d1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:46.982925 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af7b4a-1cae-46d3-b27c-6afbb2d3d1e2 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:46.983197 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af7b4a-1cae-46d3-b27c-6afbb2d3d1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:46.983461 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af7b4a-1cae-46d3-b27c-6afbb2d3d1e2 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:46.984111 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af7b4a-1cae-46d3-b27c-6afbb2d3d1e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:46:46.984410 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af7b4a-1cae-46d3-b27c-6afbb2d3d1e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:46.985754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af7b4a-1cae-46d3-b27c-6afbb2d3d1e2 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:46.990756 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:46.994277 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 593/1189] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:46:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:47.000071 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258fa35e-2f88-484b-90f5-c8f5c19af7f5 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:47.025569 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:47.097540 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258fa35e-2f88-484b-90f5-c8f5c19af7f5 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:47.131598 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c6042-66a2-4c62-bcf2-2b4d297098d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:47.132215 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c6042-66a2-4c62-bcf2-2b4d297098d7 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:47.132495 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c6042-66a2-4c62-bcf2-2b4d297098d7 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:47.132769 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c6042-66a2-4c62-bcf2-2b4d297098d7 None admin] PATH_INFO: `/v3/users/ffc09add363e44fca0d12b3466898bce` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:47.134433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c6042-66a2-4c62-bcf2-2b4d297098d7 None admin] RBAC: Authorizing `identity:get_user(user_id=ffc09add363e44fca0d12b3466898bce)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:47.135869 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c6042-66a2-4c62-bcf2-2b4d297098d7 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:47.137437 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 597/1190] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:46:46 2022] GET /identity/v3/users/ffc09add363e44fca0d12b3466898bce => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:47.143192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af7b4a-1cae-46d3-b27c-6afbb2d3d1e2 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:47.241077 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af7b4a-1cae-46d3-b27c-6afbb2d3d1e2 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:47.273558 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb779d7-ebd1-4769-b838-fe675c353980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:47.274164 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb779d7-ebd1-4769-b838-fe675c353980 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:47.275019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb779d7-ebd1-4769-b838-fe675c353980 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:47.275288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb779d7-ebd1-4769-b838-fe675c353980 None admin] PATH_INFO: `/v3/projects/1dbfe238dadf49d9a8303aee60ea5797` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:47.276390 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb779d7-ebd1-4769-b838-fe675c353980 None admin] RBAC: Authorizing `identity:get_project(project_id=1dbfe238dadf49d9a8303aee60ea5797)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:47.277792 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb779d7-ebd1-4769-b838-fe675c353980 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:47.279419 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 594/1191] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:46:47 2022] GET /identity/v3/projects/1dbfe238dadf49d9a8303aee60ea5797 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:47.285304 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c6042-66a2-4c62-bcf2-2b4d297098d7 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:47.373513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:47.381713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c6042-66a2-4c62-bcf2-2b4d297098d7 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:47.416250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf660b33-32b5-4038-ab59-8df0ac445d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:47.416889 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf660b33-32b5-4038-ab59-8df0ac445d71 None admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:47.417167 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf660b33-32b5-4038-ab59-8df0ac445d71 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:47.417439 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf660b33-32b5-4038-ab59-8df0ac445d71 None admin] PATH_INFO: `/v3/projects/1dbfe238dadf49d9a8303aee60ea5797/users/ffc09add363e44fca0d12b3466898bce/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:47.419352 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf660b33-32b5-4038-ab59-8df0ac445d71 None admin] RBAC: Authorizing `identity:create_grant(project_id=1dbfe238dadf49d9a8303aee60ea5797, user_id=ffc09add363e44fca0d12b3466898bce, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:47.420765 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf660b33-32b5-4038-ab59-8df0ac445d71 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:47.480297 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 598/1192] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 31 10:46:47 2022] PUT /identity/v3/projects/1dbfe238dadf49d9a8303aee60ea5797/users/ffc09add363e44fca0d12b3466898bce/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:46:49.353385 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a93be-739c-49d0-a7fa-a55e8fe92869 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:49.353958 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a93be-739c-49d0-a7fa-a55e8fe92869 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:49.353958 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a93be-739c-49d0-a7fa-a55e8fe92869 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:49.355285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 595/1193] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:46:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:46:49.360765 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48188b-455a-44d6-b50a-d63dbde4d27f None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:49.361187 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48188b-455a-44d6-b50a-d63dbde4d27f None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:49.361292 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48188b-455a-44d6-b50a-d63dbde4d27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:49.379373 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-6f48188b-455a-44d6-b50a-d63dbde4d27f None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:46:49.380556 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-6f48188b-455a-44d6-b50a-d63dbde4d27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:46:49.445563 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 599/1194] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:46:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:46:49.717695 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a93be-739c-49d0-a7fa-a55e8fe92869 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:49.783935 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:49.817033 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a93be-739c-49d0-a7fa-a55e8fe92869 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:49.850287 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13b0aa-f87a-43b8-a048-c2b3443cfe26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:49.850937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13b0aa-f87a-43b8-a048-c2b3443cfe26 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:49.851211 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13b0aa-f87a-43b8-a048-c2b3443cfe26 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:49.851477 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13b0aa-f87a-43b8-a048-c2b3443cfe26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:49.861827 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13b0aa-f87a-43b8-a048-c2b3443cfe26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:46:49.866574 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-8e13b0aa-f87a-43b8-a048-c2b3443cfe26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:46:49.866574 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:46:49.866574 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:46:49.866574 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:46:49.866574 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:46:49.866574 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:46:49.866574 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:46:49.866574 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:46:49.866574 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:46:49.866574 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:46:49.866574 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:46:49.866574 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:46:49.866574 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:46:49.866574 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:46:49.866574 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:46:49.866574 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:46:49.867679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:46:49.867679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:46:49.867679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:46:49.867679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:46:49.867679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:46:49.867679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:46:49.867679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:46:49.867679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:46:49.867679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:46:49.867679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:46:49.867679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:46:49.867679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:46:49.867679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:46:49.867679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:46:49.867679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:46:49.867679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:46:49.869629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:46:49.869629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:46:49.869629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:46:49.869629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:46:49.869629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:46:49.869629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:46:49.869629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:46:49.869629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:46:49.869629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:46:49.869629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:46:49.869629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:46:49.869629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:46:49.869629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 596/1195] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:46:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:46:49.874519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48188b-455a-44d6-b50a-d63dbde4d27f None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:49.951975 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:49.969386 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48188b-455a-44d6-b50a-d63dbde4d27f None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:50.003174 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a29089-00f2-4ffa-ac1d-f49c1646c439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:50.003767 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a29089-00f2-4ffa-ac1d-f49c1646c439 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:50.004040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a29089-00f2-4ffa-ac1d-f49c1646c439 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:50.004306 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a29089-00f2-4ffa-ac1d-f49c1646c439 None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:50.004998 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a29089-00f2-4ffa-ac1d-f49c1646c439 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:46:50.005295 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a29089-00f2-4ffa-ac1d-f49c1646c439 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:50.006682 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a29089-00f2-4ffa-ac1d-f49c1646c439 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:50.015051 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 600/1196] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:46:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:50.020804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13b0aa-f87a-43b8-a048-c2b3443cfe26 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:50.119530 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13b0aa-f87a-43b8-a048-c2b3443cfe26 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:50.144167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:50.151691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b1a0a-1ed4-4756-ade0-72dc1686d1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:50.152308 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b1a0a-1ed4-4756-ade0-72dc1686d1c3 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:50.155793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b1a0a-1ed4-4756-ade0-72dc1686d1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:50.155793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b1a0a-1ed4-4756-ade0-72dc1686d1c3 None admin] PATH_INFO: `/v3/users/ffc09add363e44fca0d12b3466898bce` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:50.155793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16b1a0a-1ed4-4756-ade0-72dc1686d1c3 None admin] RBAC: Authorizing `identity:get_user(user_id=ffc09add363e44fca0d12b3466898bce)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:50.155793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16b1a0a-1ed4-4756-ade0-72dc1686d1c3 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:50.156901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 597/1197] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 31 10:46:50 2022] GET /identity/v3/users/ffc09add363e44fca0d12b3466898bce => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:50.162876 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a29089-00f2-4ffa-ac1d-f49c1646c439 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:50.261406 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a29089-00f2-4ffa-ac1d-f49c1646c439 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:50.294326 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdcec4d-69a4-4250-8a6a-c16dc9ae8b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:50.294988 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdcec4d-69a4-4250-8a6a-c16dc9ae8b4a None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:50.295256 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdcec4d-69a4-4250-8a6a-c16dc9ae8b4a None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:50.295523 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdcec4d-69a4-4250-8a6a-c16dc9ae8b4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:50.310050 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:50.369907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdcec4d-69a4-4250-8a6a-c16dc9ae8b4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:50.371390 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdcec4d-69a4-4250-8a6a-c16dc9ae8b4a None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:50.465606 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 601/1198] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:46:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:46:50.471537 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b1a0a-1ed4-4756-ade0-72dc1686d1c3 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:50.569171 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b1a0a-1ed4-4756-ade0-72dc1686d1c3 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:50.601406 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25d638-586d-417d-853a-456b4a4a022f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:50.602017 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25d638-586d-417d-853a-456b4a4a022f None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:50.602291 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25d638-586d-417d-853a-456b4a4a022f None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:50.603297 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25d638-586d-417d-853a-456b4a4a022f None admin] PATH_INFO: `/v3/projects/1dbfe238dadf49d9a8303aee60ea5797` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:50.604825 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25d638-586d-417d-853a-456b4a4a022f None admin] RBAC: Authorizing `identity:get_project(project_id=1dbfe238dadf49d9a8303aee60ea5797)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:50.606249 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25d638-586d-417d-853a-456b4a4a022f None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:50.607902 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 598/1199] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 31 10:46:50 2022] GET /identity/v3/projects/1dbfe238dadf49d9a8303aee60ea5797 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:46:50.613988 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdcec4d-69a4-4250-8a6a-c16dc9ae8b4a None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:46:50.657931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:46:50.710212 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdcec4d-69a4-4250-8a6a-c16dc9ae8b4a None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:46:50.743579 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1861a0-a8f8-4e0e-aaf7-f93f64d6c022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:46:50.744166 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1861a0-a8f8-4e0e-aaf7-f93f64d6c022 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:46:50.744442 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1861a0-a8f8-4e0e-aaf7-f93f64d6c022 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:46:50.744712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1861a0-a8f8-4e0e-aaf7-f93f64d6c022 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:46:50.745422 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1861a0-a8f8-4e0e-aaf7-f93f64d6c022 None admin] RBAC: Adding query filter params (role.id=13ca33d694c645d6920930d39dc52a0d, scope.project.id=1dbfe238dadf49d9a8303aee60ea5797, user.id=ffc09add363e44fca0d12b3466898bce) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:46:50.745725 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1861a0-a8f8-4e0e-aaf7-f93f64d6c022 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:46:50.747177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1861a0-a8f8-4e0e-aaf7-f93f64d6c022 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:46:50.752833 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 602/1200] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:46:50 2022] GET /identity/v3/role_assignments?user.id=ffc09add363e44fca0d12b3466898bce&scope.project.id=1dbfe238dadf49d9a8303aee60ea5797&role.id=13ca33d694c645d6920930d39dc52a0d => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:47:00.597325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=68 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:47:00.750249 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=46 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:04.748327 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd2e5b-7339-428d-a465-d744c2ceb9c6 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:04.748945 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd2e5b-7339-428d-a465-d744c2ceb9c6 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:04.748945 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd2e5b-7339-428d-a465-d744c2ceb9c6 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:04.750245 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 599/1201] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:48:04.755351 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40899294-b9ac-4ff8-9440-09fc1a494448 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:04.755803 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40899294-b9ac-4ff8-9440-09fc1a494448 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:04.755910 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40899294-b9ac-4ff8-9440-09fc1a494448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:04.774893 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-40899294-b9ac-4ff8-9440-09fc1a494448 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:48:04.776128 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-40899294-b9ac-4ff8-9440-09fc1a494448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:48:04.844503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 603/1202] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:48:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:48:05.117817 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd2e5b-7339-428d-a465-d744c2ceb9c6 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:05.220406 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd2e5b-7339-428d-a465-d744c2ceb9c6 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:05.255256 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c40fb-835a-457d-91df-2bb70d9a0fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:05.255888 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c40fb-835a-457d-91df-2bb70d9a0fb1 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:05.256174 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c40fb-835a-457d-91df-2bb70d9a0fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:05.256444 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c40fb-835a-457d-91df-2bb70d9a0fb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:05.267079 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c40fb-835a-457d-91df-2bb70d9a0fb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:05.268513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c40fb-835a-457d-91df-2bb70d9a0fb1 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:05.284235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-328c40fb-835a-457d-91df-2bb70d9a0fb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:48:05.284235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:05.284235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:05.284235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:05.284235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:05.284235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:05.284235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:05.284235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:05.284235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:05.284235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:05.284235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:05.284235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:05.284235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:48:05.284235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:48:05.284235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:48:05.284235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:48:05.285498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:05.285498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:05.285498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:05.285498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:05.285498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:05.285498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:05.285498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:05.285498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:05.285498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:05.285498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:05.285498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:05.285498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:05.285498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:05.285498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:05.285498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:05.285498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:05.285498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:48:05.286701 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:48:05.286701 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:48:05.286701 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:48:05.286701 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 600/1203] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:48:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:05.291354 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40899294-b9ac-4ff8-9440-09fc1a494448 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:05.396325 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40899294-b9ac-4ff8-9440-09fc1a494448 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:05.424103 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:05.429837 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6c558-6c71-4ac3-931d-1b2592f49dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:05.430463 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6c558-6c71-4ac3-931d-1b2592f49dfe None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:05.430744 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6c558-6c71-4ac3-931d-1b2592f49dfe None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:05.431015 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6c558-6c71-4ac3-931d-1b2592f49dfe None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:05.431647 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6c558-6c71-4ac3-931d-1b2592f49dfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:05.431946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6c558-6c71-4ac3-931d-1b2592f49dfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:05.433340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6c558-6c71-4ac3-931d-1b2592f49dfe None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:05.445796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 604/1204] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:48:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:05.452623 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c40fb-835a-457d-91df-2bb70d9a0fb1 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:05.515426 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:05.564924 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c40fb-835a-457d-91df-2bb70d9a0fb1 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:05.599847 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac17c1e-7cba-405e-adbe-f528f4143e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:05.600447 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac17c1e-7cba-405e-adbe-f528f4143e14 None admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:05.600721 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac17c1e-7cba-405e-adbe-f528f4143e14 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:05.600989 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac17c1e-7cba-405e-adbe-f528f4143e14 None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:05.601634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac17c1e-7cba-405e-adbe-f528f4143e14 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:05.603093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac17c1e-7cba-405e-adbe-f528f4143e14 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:05.679200 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 601/1205] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 31 10:48:05 2022] POST /identity/v3/users => generated 251 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 10:48:07.613772 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6326f8-cc16-4210-88a8-60db883deb74 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:07.614335 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6326f8-cc16-4210-88a8-60db883deb74 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:07.614335 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6326f8-cc16-4210-88a8-60db883deb74 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:07.615689 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 605/1206] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:48:07.621297 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7a8aa-a7ac-4f7e-a6fe-31a3fa53892f None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:07.624551 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7a8aa-a7ac-4f7e-a6fe-31a3fa53892f None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:07.624551 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7a8aa-a7ac-4f7e-a6fe-31a3fa53892f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:07.641239 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-32b7a8aa-a7ac-4f7e-a6fe-31a3fa53892f None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:48:07.642441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-32b7a8aa-a7ac-4f7e-a6fe-31a3fa53892f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:48:07.707438 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 602/1207] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:48:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:48:07.976872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6326f8-cc16-4210-88a8-60db883deb74 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:08.078939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6326f8-cc16-4210-88a8-60db883deb74 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:08.113111 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e40840-ce94-41d0-93ac-36b4be80a3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:08.113739 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e40840-ce94-41d0-93ac-36b4be80a3ae None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:08.114014 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e40840-ce94-41d0-93ac-36b4be80a3ae None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:08.114285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e40840-ce94-41d0-93ac-36b4be80a3ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:08.125112 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e40840-ce94-41d0-93ac-36b4be80a3ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:48:08.129864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-07e40840-ce94-41d0-93ac-36b4be80a3ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:48:08.129864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:08.129864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:08.129864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:08.129864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:08.129864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:08.129864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:08.129864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:08.129864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:08.129864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:08.129864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:08.129864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:08.129864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:48:08.129864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:48:08.129864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:48:08.129864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:48:08.130924 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:48:08.130924 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:48:08.130924 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:48:08.130924 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:48:08.130924 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:08.130924 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:08.130924 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:08.130924 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:08.130924 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:08.130924 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:08.130924 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:08.130924 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:08.130924 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:08.130924 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:08.130924 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:08.130924 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:08.132252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:08.132252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:08.132252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:08.132252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:08.132252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:48:08.132252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:48:08.132252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:48:08.132252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:48:08.132252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:48:08.132252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:48:08.132252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:48:08.132252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:48:08.132252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 606/1208] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 31 10:48:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:08.136974 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b7a8aa-a7ac-4f7e-a6fe-31a3fa53892f None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:08.178566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:08.236591 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b7a8aa-a7ac-4f7e-a6fe-31a3fa53892f None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:08.269563 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81ac02-6f4f-4d5c-ac78-76222b640925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:08.270168 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81ac02-6f4f-4d5c-ac78-76222b640925 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:08.270472 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81ac02-6f4f-4d5c-ac78-76222b640925 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:08.270745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81ac02-6f4f-4d5c-ac78-76222b640925 None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:08.271404 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81ac02-6f4f-4d5c-ac78-76222b640925 None admin] RBAC: Adding query filter params (name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:08.271703 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81ac02-6f4f-4d5c-ac78-76222b640925 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:08.273064 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81ac02-6f4f-4d5c-ac78-76222b640925 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:08.280859 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 603/1209] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 31 10:48:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:08.286632 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e40840-ce94-41d0-93ac-36b4be80a3ae None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:08.339058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:08.390892 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e40840-ce94-41d0-93ac-36b4be80a3ae None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:08.423159 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c289dd-7688-4452-87e9-9776719838b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:08.423772 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c289dd-7688-4452-87e9-9776719838b1 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:08.424045 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c289dd-7688-4452-87e9-9776719838b1 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:08.424312 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c289dd-7688-4452-87e9-9776719838b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:08.434304 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c289dd-7688-4452-87e9-9776719838b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:08.436277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c289dd-7688-4452-87e9-9776719838b1 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:08.449992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-96c289dd-7688-4452-87e9-9776719838b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:48:08.449992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:08.449992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:08.449992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:08.449992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:08.449992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:08.449992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:08.449992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:08.449992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:08.449992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:08.449992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:08.449992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:08.449992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:48:08.449992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:48:08.449992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:48:08.449992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:48:08.458196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:08.458196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:08.458196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:08.458196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:08.458196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:08.458196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:08.458196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:08.458196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:08.458196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:08.458196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:08.458196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:08.458196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:08.458196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:08.458196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:08.458196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:08.458196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:08.458196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:48:08.459410 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:48:08.459410 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:48:08.459410 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:48:08.459410 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 607/1210] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:48:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:08.459687 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81ac02-6f4f-4d5c-ac78-76222b640925 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:08.552517 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:08.555332 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81ac02-6f4f-4d5c-ac78-76222b640925 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:08.590135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71500ddd-83a5-4332-b449-622535a4538f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:08.590773 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71500ddd-83a5-4332-b449-622535a4538f None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:08.591054 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71500ddd-83a5-4332-b449-622535a4538f None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:08.591448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71500ddd-83a5-4332-b449-622535a4538f None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:08.591992 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71500ddd-83a5-4332-b449-622535a4538f None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:08.592295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71500ddd-83a5-4332-b449-622535a4538f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:08.593657 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71500ddd-83a5-4332-b449-622535a4538f None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:08.604728 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 604/1211] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:48:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:08.610913 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c289dd-7688-4452-87e9-9776719838b1 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:08.710242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c289dd-7688-4452-87e9-9776719838b1 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:08.711784 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:08.743840 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd95d3d-3a14-492c-9470-de3c588a9898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:08.744455 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd95d3d-3a14-492c-9470-de3c588a9898 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:08.744732 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd95d3d-3a14-492c-9470-de3c588a9898 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:08.745001 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd95d3d-3a14-492c-9470-de3c588a9898 None admin] PATH_INFO: `/v3/users/placement` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:08.753404 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd95d3d-3a14-492c-9470-de3c588a9898 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:08.754991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd95d3d-3a14-492c-9470-de3c588a9898 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:08.771575 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-2dd95d3d-3a14-492c-9470-de3c588a9898 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 10:48:08.771575 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:08.771575 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:08.771575 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:08.771575 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:08.771575 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:08.771575 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:08.771575 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:08.771575 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:08.771575 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:08.771575 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:08.771575 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:08.771575 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:48:08.771575 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:48:08.771575 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:48:08.771575 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:48:08.778844 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:08.778844 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:08.778844 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:48:08.778844 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:48:08.778844 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:48:08.778844 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:48:08.778844 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:08.778844 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:08.778844 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:08.778844 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:08.778844 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:08.778844 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:08.778844 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:08.778844 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:08.778844 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:08.778844 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:08.778844 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:08.778844 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:08.780015 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:08.780015 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:08.780015 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:48:08.780015 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:48:08.780015 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:48:08.780015 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:48:08.780015 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:48:08.780015 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:48:08.780015 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 10:48:08.780015 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:48:08.780015 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 608/1212] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:48:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:08.780789 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71500ddd-83a5-4332-b449-622535a4538f None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:08.881681 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71500ddd-83a5-4332-b449-622535a4538f None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:08.915850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87426974-a7c1-400c-87e8-17deec2d81bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:08.916490 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87426974-a7c1-400c-87e8-17deec2d81bd None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:08.916757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87426974-a7c1-400c-87e8-17deec2d81bd None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:08.917023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87426974-a7c1-400c-87e8-17deec2d81bd None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:08.917752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87426974-a7c1-400c-87e8-17deec2d81bd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:08.918052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87426974-a7c1-400c-87e8-17deec2d81bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:08.919684 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87426974-a7c1-400c-87e8-17deec2d81bd None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:08.937575 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 605/1213] 10.222.0.16 () {56 vars in 1143 bytes} [Wed Aug 31 10:48:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:08.943749 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd95d3d-3a14-492c-9470-de3c588a9898 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:09.043672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd95d3d-3a14-492c-9470-de3c588a9898 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:09.077747 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed13b570-3a60-4fde-94c5-9fb5192213ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:09.078343 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13b570-3a60-4fde-94c5-9fb5192213ca None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:09.078653 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13b570-3a60-4fde-94c5-9fb5192213ca None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:09.078924 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13b570-3a60-4fde-94c5-9fb5192213ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:09.107817 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:09.145692 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed13b570-3a60-4fde-94c5-9fb5192213ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:09.147109 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed13b570-3a60-4fde-94c5-9fb5192213ca None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:09.240655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 609/1214] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:48:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:48:09.246880 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87426974-a7c1-400c-87e8-17deec2d81bd None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:09.266984 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:09.347478 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87426974-a7c1-400c-87e8-17deec2d81bd None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:09.380365 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a4178-6015-4990-9ff4-8ed5aa73b9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:09.380982 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a4178-6015-4990-9ff4-8ed5aa73b9db None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:09.381255 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a4178-6015-4990-9ff4-8ed5aa73b9db None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:09.381516 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a4178-6015-4990-9ff4-8ed5aa73b9db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:09.392598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a4178-6015-4990-9ff4-8ed5aa73b9db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:09.397807 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a4178-6015-4990-9ff4-8ed5aa73b9db None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:09.416938 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-367a4178-6015-4990-9ff4-8ed5aa73b9db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:48:09.416938 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:09.416938 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:09.416938 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:09.416938 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:09.416938 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:09.416938 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:09.416938 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:09.416938 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:09.416938 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:09.416938 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:09.416938 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:09.416938 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:48:09.416938 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:48:09.416938 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:48:09.416938 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:48:09.418052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:09.418052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:09.418052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:09.418052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:09.418052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:09.418052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:09.418052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:09.418052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:09.418052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:09.418052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:09.418052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:09.418052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:09.418052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:09.418052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:09.418052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:09.418052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:09.418052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:48:09.420163 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:48:09.420163 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:48:09.420163 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:48:09.420163 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 606/1215] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:48:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:09.424881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed13b570-3a60-4fde-94c5-9fb5192213ca None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:09.484849 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:09.522727 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed13b570-3a60-4fde-94c5-9fb5192213ca None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:09.555256 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054049e-284f-4d53-be97-e5b69be2c40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:09.555837 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054049e-284f-4d53-be97-e5b69be2c40d None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:09.556112 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054049e-284f-4d53-be97-e5b69be2c40d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:09.556382 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054049e-284f-4d53-be97-e5b69be2c40d None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:09.557018 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4054049e-284f-4d53-be97-e5b69be2c40d None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:09.557325 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4054049e-284f-4d53-be97-e5b69be2c40d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:09.558720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4054049e-284f-4d53-be97-e5b69be2c40d None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:09.571407 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 610/1216] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:09.577218 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a4178-6015-4990-9ff4-8ed5aa73b9db None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:09.646466 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:09.678791 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a4178-6015-4990-9ff4-8ed5aa73b9db None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:09.711174 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f144d-36b4-4dac-8080-752a5978a028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:09.711793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f144d-36b4-4dac-8080-752a5978a028 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:09.712082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f144d-36b4-4dac-8080-752a5978a028 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:09.712332 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f144d-36b4-4dac-8080-752a5978a028 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:09.718691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f144d-36b4-4dac-8080-752a5978a028 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:09.720106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f144d-36b4-4dac-8080-752a5978a028 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:09.730782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-3e8f144d-36b4-4dac-8080-752a5978a028 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:48:09.730782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:09.730782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:09.730782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:09.730782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:09.730782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:09.730782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:09.730782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:09.730782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:09.730782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:09.730782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:09.730782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:09.730782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:48:09.730782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:48:09.730782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:48:09.732423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:48:09.732423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:09.732423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:09.732423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:09.732423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:09.732423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:09.732423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:09.732423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:09.732423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:09.732423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:09.732423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:09.732423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:09.732423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:09.732423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:09.732423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:09.732423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:09.732423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:09.733525 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:48:09.733525 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:48:09.733525 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:48:09.733525 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:48:09.733525 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:48:09.733525 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:48:09.733525 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:48:09.733525 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:48:09.733525 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 607/1217] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:48:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:09.738688 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054049e-284f-4d53-be97-e5b69be2c40d None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:09.835618 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054049e-284f-4d53-be97-e5b69be2c40d None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:09.859384 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:09.871599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df60ab-b344-4a4f-8b61-009c87b0052f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:09.872195 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df60ab-b344-4a4f-8b61-009c87b0052f None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:09.872465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df60ab-b344-4a4f-8b61-009c87b0052f None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:09.873254 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df60ab-b344-4a4f-8b61-009c87b0052f None admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:09.873936 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2df60ab-b344-4a4f-8b61-009c87b0052f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:09.874234 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2df60ab-b344-4a4f-8b61-009c87b0052f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:09.875732 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2df60ab-b344-4a4f-8b61-009c87b0052f None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:09.886950 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 611/1218] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:48:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:09.893796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f144d-36b4-4dac-8080-752a5978a028 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:09.994199 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f144d-36b4-4dac-8080-752a5978a028 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:10.022448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:10.028199 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906019d3-82d0-43e2-bd3c-ef6d7ee05be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:10.028851 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906019d3-82d0-43e2-bd3c-ef6d7ee05be7 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:10.029323 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906019d3-82d0-43e2-bd3c-ef6d7ee05be7 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:10.029656 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906019d3-82d0-43e2-bd3c-ef6d7ee05be7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:10.030378 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906019d3-82d0-43e2-bd3c-ef6d7ee05be7 None admin] RBAC: Adding query filter params (role.id=bec755d6565a41798b09f5f57c1d51dd, scope.project.id=8db9b3f8a9f041938ac7b39003a35015, user.id=f3db46a4b4e74cc0bb2a7bd6f0e99105) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:10.030716 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906019d3-82d0-43e2-bd3c-ef6d7ee05be7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:10.032134 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906019d3-82d0-43e2-bd3c-ef6d7ee05be7 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:10.037372 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 608/1219] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:48:09 2022] GET /identity/v3/role_assignments?user.id=f3db46a4b4e74cc0bb2a7bd6f0e99105&scope.project.id=8db9b3f8a9f041938ac7b39003a35015&role.id=bec755d6565a41798b09f5f57c1d51dd => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:11.898580 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f155e96-ecb2-4837-98ba-2ec4bc997eba None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:11.899108 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f155e96-ecb2-4837-98ba-2ec4bc997eba None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:11.899108 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f155e96-ecb2-4837-98ba-2ec4bc997eba None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:11.900433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 612/1220] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:48:11.905717 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee366f6-68c8-44eb-ac4d-6d1165237aa6 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:11.906140 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee366f6-68c8-44eb-ac4d-6d1165237aa6 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:11.906281 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee366f6-68c8-44eb-ac4d-6d1165237aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:11.925033 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-aee366f6-68c8-44eb-ac4d-6d1165237aa6 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:48:11.926216 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-aee366f6-68c8-44eb-ac4d-6d1165237aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:48:11.993476 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 609/1221] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:48:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:48:12.262636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f155e96-ecb2-4837-98ba-2ec4bc997eba None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:12.361849 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f155e96-ecb2-4837-98ba-2ec4bc997eba None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:12.396256 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54325c6b-d35c-48c9-9008-4cbdbe666afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:12.396869 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54325c6b-d35c-48c9-9008-4cbdbe666afc None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:12.397189 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54325c6b-d35c-48c9-9008-4cbdbe666afc None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:12.397416 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54325c6b-d35c-48c9-9008-4cbdbe666afc None admin] PATH_INFO: `/v3/roles/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:12.407347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54325c6b-d35c-48c9-9008-4cbdbe666afc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:48:12.412043 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-54325c6b-d35c-48c9-9008-4cbdbe666afc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:48:12.412043 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:12.412043 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:12.412043 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:12.412043 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:12.412043 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:12.412043 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:12.412043 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:12.412043 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:12.412043 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:12.412043 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:12.412043 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:12.412043 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:48:12.412043 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:48:12.412043 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:48:12.412043 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:48:12.413154 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:48:12.413154 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:48:12.413154 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:48:12.413154 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:48:12.413154 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:12.413154 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:12.413154 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:12.413154 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:12.413154 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:12.413154 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:12.413154 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:12.413154 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:12.413154 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:12.413154 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:12.413154 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:12.413154 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:12.414238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:12.414238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:12.414238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:12.414238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:12.414238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:48:12.414238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:48:12.414238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:48:12.414238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:48:12.414238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:48:12.414238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:48:12.414238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:48:12.414238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:48:12.414238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 613/1222] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 31 10:48:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:12.419091 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee366f6-68c8-44eb-ac4d-6d1165237aa6 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:12.517617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee366f6-68c8-44eb-ac4d-6d1165237aa6 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:12.541583 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:12.552466 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2dce1b-97f6-4d73-bc8d-0ca18343688b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:12.553088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2dce1b-97f6-4d73-bc8d-0ca18343688b None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:12.553357 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2dce1b-97f6-4d73-bc8d-0ca18343688b None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:12.553723 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2dce1b-97f6-4d73-bc8d-0ca18343688b None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:12.554310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2dce1b-97f6-4d73-bc8d-0ca18343688b None admin] RBAC: Adding query filter params (name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:12.554633 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2dce1b-97f6-4d73-bc8d-0ca18343688b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:12.555975 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2dce1b-97f6-4d73-bc8d-0ca18343688b None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:12.564284 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 610/1223] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 31 10:48:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:12.569984 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54325c6b-d35c-48c9-9008-4cbdbe666afc None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:12.613327 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:12.670535 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54325c6b-d35c-48c9-9008-4cbdbe666afc None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:12.703957 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b43ac-1585-473b-a25e-37bb6124f0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:12.704571 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b43ac-1585-473b-a25e-37bb6124f0de None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:12.704846 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b43ac-1585-473b-a25e-37bb6124f0de None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:12.705115 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b43ac-1585-473b-a25e-37bb6124f0de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:12.714795 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b43ac-1585-473b-a25e-37bb6124f0de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:12.716316 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b43ac-1585-473b-a25e-37bb6124f0de None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:12.730638 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-802b43ac-1585-473b-a25e-37bb6124f0de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:48:12.730638 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:12.730638 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:12.730638 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:12.730638 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:12.730638 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:12.730638 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:12.730638 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:12.730638 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:12.730638 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:12.730638 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:12.730638 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:12.730638 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:48:12.730638 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:48:12.730638 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:48:12.730638 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:48:12.731737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:12.731737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:12.731737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:12.731737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:12.731737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:12.731737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:12.731737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:12.731737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:12.731737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:12.731737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:12.731737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:12.731737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:12.731737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:12.731737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:12.731737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:12.731737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:12.731737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:48:12.733447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:48:12.733447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:48:12.733447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:48:12.733447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 614/1224] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:48:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:12.738043 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2dce1b-97f6-4d73-bc8d-0ca18343688b None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:12.836892 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2dce1b-97f6-4d73-bc8d-0ca18343688b None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:12.871176 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231aa19-22a3-42bc-993a-ab10982291e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:12.871784 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231aa19-22a3-42bc-993a-ab10982291e2 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:12.872060 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231aa19-22a3-42bc-993a-ab10982291e2 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:12.872338 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231aa19-22a3-42bc-993a-ab10982291e2 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:12.873171 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231aa19-22a3-42bc-993a-ab10982291e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:12.873485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231aa19-22a3-42bc-993a-ab10982291e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:12.874930 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231aa19-22a3-42bc-993a-ab10982291e2 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:12.886276 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 611/1225] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:48:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:12.892316 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b43ac-1585-473b-a25e-37bb6124f0de None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:12.985303 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:12.991502 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b43ac-1585-473b-a25e-37bb6124f0de None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:13.024430 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751edfa-3f03-42a1-bd9b-d94b84b4673a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:13.025052 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751edfa-3f03-42a1-bd9b-d94b84b4673a None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:13.025387 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751edfa-3f03-42a1-bd9b-d94b84b4673a None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:13.025665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751edfa-3f03-42a1-bd9b-d94b84b4673a None admin] PATH_INFO: `/v3/users/placement` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:13.033533 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7751edfa-3f03-42a1-bd9b-d94b84b4673a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:13.035027 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7751edfa-3f03-42a1-bd9b-d94b84b4673a None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:13.046823 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-7751edfa-3f03-42a1-bd9b-d94b84b4673a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 10:48:13.046823 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:13.046823 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:13.046823 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:13.046823 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:13.046823 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:13.046823 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:13.046823 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:13.046823 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:13.046823 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:13.046823 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:13.046823 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:13.046823 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:48:13.046823 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:48:13.046823 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:48:13.046823 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:48:13.047947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:13.047947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:13.047947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:48:13.047947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:48:13.047947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:48:13.047947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:48:13.047947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:13.047947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:13.047947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:13.047947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:13.047947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:13.047947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:13.047947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:13.047947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:13.047947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:13.047947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:13.047947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:13.047947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:13.049148 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:13.049148 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:13.049148 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:48:13.049148 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:48:13.049148 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:48:13.049148 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:48:13.049148 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:48:13.049148 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:48:13.049148 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 10:48:13.049148 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:48:13.049148 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 615/1226] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:48:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:13.053912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231aa19-22a3-42bc-993a-ab10982291e2 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:13.100998 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:13.152016 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231aa19-22a3-42bc-993a-ab10982291e2 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:13.186267 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99685e1-aefe-4958-b905-dc4fc871cd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:13.186892 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99685e1-aefe-4958-b905-dc4fc871cd42 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:13.187200 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99685e1-aefe-4958-b905-dc4fc871cd42 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:13.187515 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99685e1-aefe-4958-b905-dc4fc871cd42 None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:13.188433 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99685e1-aefe-4958-b905-dc4fc871cd42 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:13.188957 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99685e1-aefe-4958-b905-dc4fc871cd42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:13.190991 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99685e1-aefe-4958-b905-dc4fc871cd42 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:13.208667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 612/1227] 10.222.0.16 () {56 vars in 1143 bytes} [Wed Aug 31 10:48:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:13.214345 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751edfa-3f03-42a1-bd9b-d94b84b4673a None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:13.314619 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751edfa-3f03-42a1-bd9b-d94b84b4673a None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:13.348253 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d14fc-5239-45e8-995e-2a4956325790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:13.348870 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d14fc-5239-45e8-995e-2a4956325790 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:13.349148 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d14fc-5239-45e8-995e-2a4956325790 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:13.349418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d14fc-5239-45e8-995e-2a4956325790 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:13.359781 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d14fc-5239-45e8-995e-2a4956325790 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:13.361166 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d14fc-5239-45e8-995e-2a4956325790 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:13.375139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-f09d14fc-5239-45e8-995e-2a4956325790 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:48:13.375139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:13.375139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:13.375139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:13.375139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:13.375139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:13.375139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:13.375139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:13.375139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:13.375139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:13.375139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:13.375139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:13.375139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:48:13.375139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:48:13.375139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:48:13.375139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:48:13.380874 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:13.380874 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:13.380874 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:13.380874 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:13.380874 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:13.380874 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:13.380874 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:13.380874 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:13.380874 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:13.380874 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:13.380874 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:13.380874 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:13.380874 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:13.380874 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:13.380874 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:13.380874 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:13.380874 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:48:13.381941 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:48:13.381941 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:48:13.381941 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:48:13.381941 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 616/1228] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:48:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:13.382289 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99685e1-aefe-4958-b905-dc4fc871cd42 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:13.481300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99685e1-aefe-4958-b905-dc4fc871cd42 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:13.483673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:13.514417 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a46c56-ed5f-4e09-b5b6-44482a8cd7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:13.515019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a46c56-ed5f-4e09-b5b6-44482a8cd7d0 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:13.515293 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a46c56-ed5f-4e09-b5b6-44482a8cd7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:13.515560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a46c56-ed5f-4e09-b5b6-44482a8cd7d0 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:13.516190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a46c56-ed5f-4e09-b5b6-44482a8cd7d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:13.516490 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a46c56-ed5f-4e09-b5b6-44482a8cd7d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:13.517839 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a46c56-ed5f-4e09-b5b6-44482a8cd7d0 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:13.529553 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 613/1229] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:13.535302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d14fc-5239-45e8-995e-2a4956325790 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:13.545745 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:13.634192 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d14fc-5239-45e8-995e-2a4956325790 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:13.666328 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b67e0e-12a4-4eb2-ad21-b3124cc9c9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:13.666972 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b67e0e-12a4-4eb2-ad21-b3124cc9c9e7 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:13.667304 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b67e0e-12a4-4eb2-ad21-b3124cc9c9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:13.667616 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b67e0e-12a4-4eb2-ad21-b3124cc9c9e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:13.674119 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b67e0e-12a4-4eb2-ad21-b3124cc9c9e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:13.675614 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b67e0e-12a4-4eb2-ad21-b3124cc9c9e7 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:13.687390 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-e9b67e0e-12a4-4eb2-ad21-b3124cc9c9e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:48:13.687390 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:13.687390 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:13.687390 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:13.687390 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:13.687390 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:13.687390 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:13.687390 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:13.687390 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:13.687390 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:13.687390 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:13.687390 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:13.687390 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:48:13.687390 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:48:13.687390 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:48:13.688480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:48:13.688480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:13.688480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:13.688480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:13.688480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:13.688480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:13.688480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:13.688480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:13.688480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:13.688480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:13.688480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:13.688480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:13.688480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:13.688480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:13.688480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:13.688480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:13.688480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:13.689785 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:48:13.689785 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:48:13.689785 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:48:13.689785 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:48:13.689785 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:48:13.689785 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:48:13.689785 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:48:13.689785 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:48:13.689785 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 617/1230] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:48:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:13.694587 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a46c56-ed5f-4e09-b5b6-44482a8cd7d0 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:13.793887 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a46c56-ed5f-4e09-b5b6-44482a8cd7d0 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:13.828307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa2bcd-b4fe-40ca-9e8b-c73d622bd1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:13.828902 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa2bcd-b4fe-40ca-9e8b-c73d622bd1fb None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:13.829176 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa2bcd-b4fe-40ca-9e8b-c73d622bd1fb None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:13.829442 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa2bcd-b4fe-40ca-9e8b-c73d622bd1fb None admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:13.830124 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa2bcd-b4fe-40ca-9e8b-c73d622bd1fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:13.830462 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa2bcd-b4fe-40ca-9e8b-c73d622bd1fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:13.831874 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa2bcd-b4fe-40ca-9e8b-c73d622bd1fb None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:13.843938 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 614/1231] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:48:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:13.849854 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b67e0e-12a4-4eb2-ad21-b3124cc9c9e7 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:13.916536 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:13.951372 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b67e0e-12a4-4eb2-ad21-b3124cc9c9e7 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:13.984432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a372c02-60c7-4e0f-88b5-8f997ea9c39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:13.985089 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a372c02-60c7-4e0f-88b5-8f997ea9c39d None admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:13.985347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a372c02-60c7-4e0f-88b5-8f997ea9c39d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:13.985880 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a372c02-60c7-4e0f-88b5-8f997ea9c39d None admin] PATH_INFO: `/v3/projects/8db9b3f8a9f041938ac7b39003a35015/users/f3db46a4b4e74cc0bb2a7bd6f0e99105/roles/bec755d6565a41798b09f5f57c1d51dd` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:14.000595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a372c02-60c7-4e0f-88b5-8f997ea9c39d None admin] RBAC: Authorizing `identity:create_grant(project_id=8db9b3f8a9f041938ac7b39003a35015, user_id=f3db46a4b4e74cc0bb2a7bd6f0e99105, role_id=bec755d6565a41798b09f5f57c1d51dd)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:14.002009 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a372c02-60c7-4e0f-88b5-8f997ea9c39d None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:14.159400 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 618/1232] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 31 10:48:13 2022] PUT /identity/v3/projects/8db9b3f8a9f041938ac7b39003a35015/users/f3db46a4b4e74cc0bb2a7bd6f0e99105/roles/bec755d6565a41798b09f5f57c1d51dd => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:48:16.063505 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea054b9e-6408-4481-88b8-55d34296962c None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:16.064083 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea054b9e-6408-4481-88b8-55d34296962c None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:16.064083 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea054b9e-6408-4481-88b8-55d34296962c None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:16.065402 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 615/1233] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:48:16.070669 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85c63a-3ede-43b4-a251-1ff21c9ca2ed None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:16.071362 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85c63a-3ede-43b4-a251-1ff21c9ca2ed None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:16.071362 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85c63a-3ede-43b4-a251-1ff21c9ca2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:16.090069 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-be85c63a-3ede-43b4-a251-1ff21c9ca2ed None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:48:16.091366 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-be85c63a-3ede-43b4-a251-1ff21c9ca2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:48:16.159397 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 619/1234] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:48:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:48:16.425459 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea054b9e-6408-4481-88b8-55d34296962c None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:16.450224 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:16.527210 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea054b9e-6408-4481-88b8-55d34296962c None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:16.561563 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09394b4c-befc-4f9b-af06-7084f53b8389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:16.562174 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09394b4c-befc-4f9b-af06-7084f53b8389 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:16.562471 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09394b4c-befc-4f9b-af06-7084f53b8389 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:16.562746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09394b4c-befc-4f9b-af06-7084f53b8389 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:16.572694 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09394b4c-befc-4f9b-af06-7084f53b8389 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:48:16.577457 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-09394b4c-befc-4f9b-af06-7084f53b8389 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:48:16.577457 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:16.577457 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:16.577457 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:16.577457 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:16.577457 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:16.577457 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:16.577457 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:16.577457 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:16.577457 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:16.577457 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:16.577457 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:16.577457 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:48:16.577457 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:48:16.577457 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:48:16.577457 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:48:16.578630 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:48:16.578630 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:48:16.578630 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:48:16.578630 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:48:16.578630 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:16.578630 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:16.578630 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:16.578630 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:16.578630 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:16.578630 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:16.578630 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:16.578630 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:16.578630 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:16.578630 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:16.578630 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:16.578630 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:16.579651 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:16.579651 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:16.579651 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:16.579651 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:16.579651 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:48:16.579651 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:48:16.579651 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:48:16.579651 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:48:16.579651 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:48:16.579651 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:48:16.579651 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:48:16.579651 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:48:16.579651 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 616/1235] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 31 10:48:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:16.584693 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be85c63a-3ede-43b4-a251-1ff21c9ca2ed None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:16.631400 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:16.683714 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be85c63a-3ede-43b4-a251-1ff21c9ca2ed None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:16.717003 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff4d7b-789f-4bab-b52f-cb5ca392e4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:16.717598 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff4d7b-789f-4bab-b52f-cb5ca392e4d9 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:16.717869 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff4d7b-789f-4bab-b52f-cb5ca392e4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:16.718175 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff4d7b-789f-4bab-b52f-cb5ca392e4d9 None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:16.718893 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff4d7b-789f-4bab-b52f-cb5ca392e4d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:16.719189 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff4d7b-789f-4bab-b52f-cb5ca392e4d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:16.720539 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff4d7b-789f-4bab-b52f-cb5ca392e4d9 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:16.728261 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 620/1236] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 31 10:48:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:16.734169 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09394b4c-befc-4f9b-af06-7084f53b8389 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:16.818751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:16.834931 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09394b4c-befc-4f9b-af06-7084f53b8389 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:16.870530 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3834d-6d72-40de-98e8-4cbb0e67573c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:16.871160 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3834d-6d72-40de-98e8-4cbb0e67573c None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:16.871437 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3834d-6d72-40de-98e8-4cbb0e67573c None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:16.871708 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3834d-6d72-40de-98e8-4cbb0e67573c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:16.882121 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3834d-6d72-40de-98e8-4cbb0e67573c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:16.883591 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3834d-6d72-40de-98e8-4cbb0e67573c None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:16.898569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-1ed3834d-6d72-40de-98e8-4cbb0e67573c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:48:16.898569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:16.898569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:16.898569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:16.898569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:16.898569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:16.898569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:16.898569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:16.898569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:16.898569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:16.898569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:16.898569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:16.898569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:48:16.898569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:48:16.898569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:48:16.898569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:48:16.899667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:16.899667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:16.899667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:16.899667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:16.899667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:16.899667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:16.899667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:16.899667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:16.899667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:16.899667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:16.899667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:16.899667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:16.899667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:16.899667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:16.899667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:16.899667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:16.899667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:48:16.901017 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:48:16.901017 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:48:16.901017 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:48:16.901017 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 617/1237] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:48:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:16.905889 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff4d7b-789f-4bab-b52f-cb5ca392e4d9 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:17.005827 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff4d7b-789f-4bab-b52f-cb5ca392e4d9 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:17.014854 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:17.038317 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da516411-2f09-4319-bce4-d8e5e0d2fb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:17.038943 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da516411-2f09-4319-bce4-d8e5e0d2fb3d None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:17.039221 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da516411-2f09-4319-bce4-d8e5e0d2fb3d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:17.039488 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da516411-2f09-4319-bce4-d8e5e0d2fb3d None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:17.040145 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da516411-2f09-4319-bce4-d8e5e0d2fb3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:17.040418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da516411-2f09-4319-bce4-d8e5e0d2fb3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:17.041777 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da516411-2f09-4319-bce4-d8e5e0d2fb3d None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:17.054009 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 621/1238] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:48:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:17.059744 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3834d-6d72-40de-98e8-4cbb0e67573c None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:17.161693 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3834d-6d72-40de-98e8-4cbb0e67573c None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:17.195922 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39098d-fc37-48c3-9bb0-ac5c938f9e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:17.196540 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39098d-fc37-48c3-9bb0-ac5c938f9e22 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:17.196817 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39098d-fc37-48c3-9bb0-ac5c938f9e22 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:17.197081 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39098d-fc37-48c3-9bb0-ac5c938f9e22 None admin] PATH_INFO: `/v3/users/placement` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:17.199758 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:17.204860 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed39098d-fc37-48c3-9bb0-ac5c938f9e22 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:17.206283 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed39098d-fc37-48c3-9bb0-ac5c938f9e22 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:17.217325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-ed39098d-fc37-48c3-9bb0-ac5c938f9e22 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 10:48:17.217325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:17.217325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:17.217325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:17.217325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:17.217325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:17.217325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:17.217325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:17.217325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:17.217325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:17.217325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:17.217325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:17.217325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:48:17.217325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:48:17.217325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:48:17.217325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:48:17.218500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:17.218500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:17.218500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:48:17.218500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:48:17.218500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:48:17.218500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:48:17.218500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:17.218500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:17.218500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:17.218500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:17.218500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:17.218500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:17.218500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:17.218500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:17.218500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:17.218500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:17.218500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:17.218500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:17.219831 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:17.219831 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:17.219831 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:48:17.219831 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:48:17.219831 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:48:17.219831 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:48:17.219831 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:48:17.219831 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:48:17.219831 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 10:48:17.219831 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:48:17.219831 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 618/1239] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:48:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:17.224410 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da516411-2f09-4319-bce4-d8e5e0d2fb3d None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:17.326618 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da516411-2f09-4319-bce4-d8e5e0d2fb3d None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:17.366291 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6dc92a-9540-4a4e-a889-33375e57f698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:17.367564 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dc92a-9540-4a4e-a889-33375e57f698 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:17.368023 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dc92a-9540-4a4e-a889-33375e57f698 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:17.368467 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dc92a-9540-4a4e-a889-33375e57f698 None admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:17.369610 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6dc92a-9540-4a4e-a889-33375e57f698 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:17.370094 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6dc92a-9540-4a4e-a889-33375e57f698 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:17.371616 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6dc92a-9540-4a4e-a889-33375e57f698 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:17.393294 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 622/1240] 10.222.0.16 () {56 vars in 1143 bytes} [Wed Aug 31 10:48:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:17.399314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39098d-fc37-48c3-9bb0-ac5c938f9e22 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:17.501366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39098d-fc37-48c3-9bb0-ac5c938f9e22 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:17.536371 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce49ee-b311-48eb-86c8-6aa9fdd37922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:17.536979 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce49ee-b311-48eb-86c8-6aa9fdd37922 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:17.537250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce49ee-b311-48eb-86c8-6aa9fdd37922 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:17.537517 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce49ee-b311-48eb-86c8-6aa9fdd37922 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:17.601621 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:17.608963 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce49ee-b311-48eb-86c8-6aa9fdd37922 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:17.610439 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce49ee-b311-48eb-86c8-6aa9fdd37922 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:17.708150 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 619/1241] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:48:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:48:17.714134 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6dc92a-9540-4a4e-a889-33375e57f698 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:17.743098 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:17.816534 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6dc92a-9540-4a4e-a889-33375e57f698 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:17.853025 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a6562-14bd-456c-bc6f-5725222657cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:17.853583 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a6562-14bd-456c-bc6f-5725222657cf None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:17.853854 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a6562-14bd-456c-bc6f-5725222657cf None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:17.854121 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a6562-14bd-456c-bc6f-5725222657cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:17.865504 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a6562-14bd-456c-bc6f-5725222657cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:17.867329 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a6562-14bd-456c-bc6f-5725222657cf None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:17.882160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-965a6562-14bd-456c-bc6f-5725222657cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:48:17.882160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:17.882160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:17.882160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:17.882160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:17.882160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:17.882160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:17.882160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:17.882160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:17.882160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:17.882160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:17.882160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:17.882160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:48:17.882160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:48:17.882160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:48:17.882160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:48:17.886275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:17.886275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:17.886275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:17.886275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:17.886275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:17.886275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:17.886275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:17.886275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:17.886275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:17.886275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:17.886275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:17.886275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:17.886275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:17.886275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:17.886275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:17.886275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:17.886275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:48:17.887417 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:48:17.887417 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:48:17.887417 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:48:17.887417 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 623/1242] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:48:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:17.890185 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce49ee-b311-48eb-86c8-6aa9fdd37922 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:17.987640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:17.993625 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce49ee-b311-48eb-86c8-6aa9fdd37922 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:18.026754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed2c8e-933b-468a-a1ec-0f5c195af766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:18.027393 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed2c8e-933b-468a-a1ec-0f5c195af766 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:18.027670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed2c8e-933b-468a-a1ec-0f5c195af766 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:18.027943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed2c8e-933b-468a-a1ec-0f5c195af766 None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:18.029184 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed2c8e-933b-468a-a1ec-0f5c195af766 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:18.029581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed2c8e-933b-468a-a1ec-0f5c195af766 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:18.031193 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed2c8e-933b-468a-a1ec-0f5c195af766 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:18.042177 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 620/1243] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:48:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:48:18.048398 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a6562-14bd-456c-bc6f-5725222657cf None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:18.129415 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:18.157168 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a6562-14bd-456c-bc6f-5725222657cf None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:18.204599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18562aa7-faec-472e-bf14-7728595ff1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:18.205210 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18562aa7-faec-472e-bf14-7728595ff1a0 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:18.205553 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18562aa7-faec-472e-bf14-7728595ff1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:18.205825 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18562aa7-faec-472e-bf14-7728595ff1a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:18.212195 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18562aa7-faec-472e-bf14-7728595ff1a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:18.213616 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18562aa7-faec-472e-bf14-7728595ff1a0 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:18.225599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-18562aa7-faec-472e-bf14-7728595ff1a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:48:18.225599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:18.225599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:18.225599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:18.225599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:18.225599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:18.225599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:18.225599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:18.225599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:18.225599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:18.225599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:18.225599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:18.225599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:48:18.225599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:48:18.225599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:48:18.229990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:48:18.229990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:18.229990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:18.229990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:18.229990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:18.229990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:18.229990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:18.229990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:18.229990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:18.229990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:18.229990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:18.229990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:18.229990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:18.229990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:18.229990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:18.229990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:18.229990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:18.231140 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:48:18.231140 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:48:18.231140 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:48:18.231140 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:48:18.231140 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:48:18.231140 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:48:18.231140 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:48:18.231140 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:48:18.231140 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 624/1244] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:48:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:18.232757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed2c8e-933b-468a-a1ec-0f5c195af766 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:18.332369 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed2c8e-933b-468a-a1ec-0f5c195af766 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:18.365091 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f8317-be73-44d8-a9ff-33d5b6f559fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:18.365627 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f8317-be73-44d8-a9ff-33d5b6f559fc None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:18.365906 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f8317-be73-44d8-a9ff-33d5b6f559fc None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:18.366179 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f8317-be73-44d8-a9ff-33d5b6f559fc None admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:18.366893 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f8317-be73-44d8-a9ff-33d5b6f559fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:18.367449 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f8317-be73-44d8-a9ff-33d5b6f559fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:18.368882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f8317-be73-44d8-a9ff-33d5b6f559fc None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:18.381177 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 621/1245] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:48:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:18.387389 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18562aa7-faec-472e-bf14-7728595ff1a0 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:18.485480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18562aa7-faec-472e-bf14-7728595ff1a0 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:18.518542 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d3290-55a6-4eb9-8857-91fc240fa053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:18.519112 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d3290-55a6-4eb9-8857-91fc240fa053 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:18.519384 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d3290-55a6-4eb9-8857-91fc240fa053 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:18.519662 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d3290-55a6-4eb9-8857-91fc240fa053 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:18.520357 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d3290-55a6-4eb9-8857-91fc240fa053 None admin] RBAC: Adding query filter params (role.id=bec755d6565a41798b09f5f57c1d51dd, scope.project.id=8db9b3f8a9f041938ac7b39003a35015, user.id=f3db46a4b4e74cc0bb2a7bd6f0e99105) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:18.520686 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d3290-55a6-4eb9-8857-91fc240fa053 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:18.522072 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d3290-55a6-4eb9-8857-91fc240fa053 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:18.527738 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 625/1246] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:48:18 2022] GET /identity/v3/role_assignments?user.id=f3db46a4b4e74cc0bb2a7bd6f0e99105&scope.project.id=8db9b3f8a9f041938ac7b39003a35015&role.id=bec755d6565a41798b09f5f57c1d51dd => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:20.473651 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1d05b-a4f6-49e1-9de5-779d4d31a873 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:20.474245 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1d05b-a4f6-49e1-9de5-779d4d31a873 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:20.474245 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1d05b-a4f6-49e1-9de5-779d4d31a873 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:20.475562 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 622/1247] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:48:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:48:20.480800 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae45121-bc62-4597-bf9d-b4f1b3fc4e0c None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:20.481196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae45121-bc62-4597-bf9d-b4f1b3fc4e0c None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:20.481344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae45121-bc62-4597-bf9d-b4f1b3fc4e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:20.488054 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:20.500792 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-4ae45121-bc62-4597-bf9d-b4f1b3fc4e0c None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:48:20.501982 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-4ae45121-bc62-4597-bf9d-b4f1b3fc4e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:48:20.567372 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 626/1248] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:48:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:48:20.838972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1d05b-a4f6-49e1-9de5-779d4d31a873 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:20.857649 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:20.941725 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1d05b-a4f6-49e1-9de5-779d4d31a873 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:20.974154 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dffe8a-57aa-4d67-a578-a97bea77119a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:20.974809 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dffe8a-57aa-4d67-a578-a97bea77119a None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:20.975083 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dffe8a-57aa-4d67-a578-a97bea77119a None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:20.975350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dffe8a-57aa-4d67-a578-a97bea77119a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:20.987049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dffe8a-57aa-4d67-a578-a97bea77119a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:48:20.991722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-24dffe8a-57aa-4d67-a578-a97bea77119a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:48:20.991722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:20.991722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:20.991722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:20.991722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:20.991722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:20.991722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:20.991722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:20.991722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:20.991722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:20.991722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:20.991722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:20.991722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:48:20.991722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:48:20.991722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:48:20.991722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:48:20.993097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:48:20.993097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:48:20.993097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:48:20.993097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:48:20.993097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:20.993097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:20.993097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:20.993097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:20.993097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:20.993097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:20.993097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:20.993097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:20.993097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:20.993097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:20.993097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:20.993097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:20.994134 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:20.994134 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:20.994134 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:20.994134 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:20.994134 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:48:20.994134 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:48:20.994134 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:48:20.994134 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:48:20.994134 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:48:20.994134 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:48:20.994134 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:48:20.994134 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:48:20.994134 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 623/1249] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:48:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:20.999885 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae45121-bc62-4597-bf9d-b4f1b3fc4e0c None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:21.098194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae45121-bc62-4597-bf9d-b4f1b3fc4e0c None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:21.128547 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:21.133482 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b58b4-13b7-487f-9b3e-79e8aef9b781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:21.134091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b58b4-13b7-487f-9b3e-79e8aef9b781 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:21.134375 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b58b4-13b7-487f-9b3e-79e8aef9b781 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:21.134674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b58b4-13b7-487f-9b3e-79e8aef9b781 None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:21.135347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b58b4-13b7-487f-9b3e-79e8aef9b781 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:21.135651 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b58b4-13b7-487f-9b3e-79e8aef9b781 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:21.137074 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b58b4-13b7-487f-9b3e-79e8aef9b781 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:21.145567 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 627/1250] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:48:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:21.152544 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dffe8a-57aa-4d67-a578-a97bea77119a None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:21.234380 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:21.257707 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dffe8a-57aa-4d67-a578-a97bea77119a None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:21.290304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65428ddf-4721-4244-ab6c-f6d1bf1c7a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:21.290946 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65428ddf-4721-4244-ab6c-f6d1bf1c7a68 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:21.291218 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65428ddf-4721-4244-ab6c-f6d1bf1c7a68 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:21.291484 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65428ddf-4721-4244-ab6c-f6d1bf1c7a68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:21.301741 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65428ddf-4721-4244-ab6c-f6d1bf1c7a68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:21.303180 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65428ddf-4721-4244-ab6c-f6d1bf1c7a68 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:21.317288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-65428ddf-4721-4244-ab6c-f6d1bf1c7a68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:48:21.317288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:21.317288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:21.317288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:21.317288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:21.317288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:21.317288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:21.317288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:21.317288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:21.317288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:21.317288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:21.317288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:21.317288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:48:21.317288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:48:21.317288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:48:21.317288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:48:21.318511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:21.318511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:21.318511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:21.318511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:21.318511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:21.318511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:21.318511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:21.318511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:21.318511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:21.318511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:21.318511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:21.318511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:21.318511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:21.318511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:21.318511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:21.318511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:21.318511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:48:21.319583 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:48:21.319583 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:48:21.319583 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:48:21.319583 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 624/1251] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:48:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:21.324600 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b58b4-13b7-487f-9b3e-79e8aef9b781 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:21.427657 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b58b4-13b7-487f-9b3e-79e8aef9b781 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:21.462226 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33517f9-e00b-4a50-9f15-761f49c3f989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:21.462884 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33517f9-e00b-4a50-9f15-761f49c3f989 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:21.463194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33517f9-e00b-4a50-9f15-761f49c3f989 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:21.463470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33517f9-e00b-4a50-9f15-761f49c3f989 None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:21.464144 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33517f9-e00b-4a50-9f15-761f49c3f989 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:21.464447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33517f9-e00b-4a50-9f15-761f49c3f989 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:21.466007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33517f9-e00b-4a50-9f15-761f49c3f989 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:21.477960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 628/1252] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:48:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:21.484649 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65428ddf-4721-4244-ab6c-f6d1bf1c7a68 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:21.605593 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65428ddf-4721-4244-ab6c-f6d1bf1c7a68 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:21.633272 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:21.641066 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1a093-da35-4384-85a7-52428abb61b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:21.641692 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1a093-da35-4384-85a7-52428abb61b2 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:21.641967 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1a093-da35-4384-85a7-52428abb61b2 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:21.642235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1a093-da35-4384-85a7-52428abb61b2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:21.649693 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d1a093-da35-4384-85a7-52428abb61b2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:21.651300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d1a093-da35-4384-85a7-52428abb61b2 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:21.662345 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-91d1a093-da35-4384-85a7-52428abb61b2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 10:48:21.662345 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:21.662345 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:21.662345 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:21.662345 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:21.662345 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:21.662345 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:21.662345 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:21.662345 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:21.662345 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:21.662345 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:21.662345 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:21.662345 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:48:21.662345 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:48:21.662345 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:48:21.662345 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:48:21.668130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:21.668130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:21.668130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:48:21.668130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:48:21.668130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:48:21.668130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:48:21.668130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:21.668130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:21.668130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:21.668130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:21.668130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:21.668130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:21.668130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:21.668130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:21.668130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:21.668130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:21.668130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:21.668130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:21.669843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:21.669843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:21.669843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:48:21.669843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:48:21.669843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:48:21.669843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:48:21.669843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:48:21.669843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:48:21.669843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 10:48:21.669843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:48:21.669843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 625/1253] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:48:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:21.670491 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33517f9-e00b-4a50-9f15-761f49c3f989 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:21.727983 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:21.771782 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33517f9-e00b-4a50-9f15-761f49c3f989 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:21.805844 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7f08d-2930-41a1-bbe4-f5014d259f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:21.806463 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7f08d-2930-41a1-bbe4-f5014d259f65 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:21.806738 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7f08d-2930-41a1-bbe4-f5014d259f65 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:21.807005 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7f08d-2930-41a1-bbe4-f5014d259f65 None admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:21.807718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d7f08d-2930-41a1-bbe4-f5014d259f65 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:21.808019 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d7f08d-2930-41a1-bbe4-f5014d259f65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:21.809429 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d7f08d-2930-41a1-bbe4-f5014d259f65 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:21.826933 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 629/1254] 10.222.0.16 () {56 vars in 1143 bytes} [Wed Aug 31 10:48:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:48:21.832807 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1a093-da35-4384-85a7-52428abb61b2 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:21.934485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1a093-da35-4384-85a7-52428abb61b2 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:21.969572 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee48dbb-437a-4fb9-b6dd-5d8e777606a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:21.970106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee48dbb-437a-4fb9-b6dd-5d8e777606a7 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:21.970427 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee48dbb-437a-4fb9-b6dd-5d8e777606a7 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:21.970724 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee48dbb-437a-4fb9-b6dd-5d8e777606a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:22.026957 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:22.039001 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee48dbb-437a-4fb9-b6dd-5d8e777606a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:22.040447 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee48dbb-437a-4fb9-b6dd-5d8e777606a7 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:22.141863 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 626/1255] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:48:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:48:22.147923 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7f08d-2930-41a1-bbe4-f5014d259f65 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:22.246868 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7f08d-2930-41a1-bbe4-f5014d259f65 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:22.260360 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:22.280861 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a50920-96cf-4e92-aa13-b11f45c16c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:22.281422 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a50920-96cf-4e92-aa13-b11f45c16c23 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:22.281701 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a50920-96cf-4e92-aa13-b11f45c16c23 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:22.281975 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a50920-96cf-4e92-aa13-b11f45c16c23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:22.292014 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a50920-96cf-4e92-aa13-b11f45c16c23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:22.293526 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a50920-96cf-4e92-aa13-b11f45c16c23 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:22.308712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-76a50920-96cf-4e92-aa13-b11f45c16c23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:48:22.308712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:22.308712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:22.308712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:22.308712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:22.308712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:22.308712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:22.308712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:22.308712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:22.308712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:22.308712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:22.308712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:22.308712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:48:22.308712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:48:22.308712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:48:22.308712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:48:22.310301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:22.310301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:22.310301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:22.310301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:22.310301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:22.310301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:22.310301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:22.310301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:22.310301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:22.310301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:22.310301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:22.310301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:22.310301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:22.310301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:22.310301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:22.310301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:22.310301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:48:22.311226 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:48:22.311226 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:48:22.311226 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:48:22.311226 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 630/1256] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:48:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:22.316236 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee48dbb-437a-4fb9-b6dd-5d8e777606a7 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:22.403779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:22.415865 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee48dbb-437a-4fb9-b6dd-5d8e777606a7 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:22.449433 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68cfb1e-3c13-4ddb-ae95-0184b13adecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:22.450078 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cfb1e-3c13-4ddb-ae95-0184b13adecd None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:22.450410 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cfb1e-3c13-4ddb-ae95-0184b13adecd None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:22.450848 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cfb1e-3c13-4ddb-ae95-0184b13adecd None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:22.451492 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68cfb1e-3c13-4ddb-ae95-0184b13adecd None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:22.451795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68cfb1e-3c13-4ddb-ae95-0184b13adecd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:22.453157 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68cfb1e-3c13-4ddb-ae95-0184b13adecd None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:22.464890 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 627/1257] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:22.470762 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a50920-96cf-4e92-aa13-b11f45c16c23 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:22.585048 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a50920-96cf-4e92-aa13-b11f45c16c23 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:22.620811 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56829b6f-bf72-4de7-b507-5563e461811d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:22.621431 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56829b6f-bf72-4de7-b507-5563e461811d None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:22.621710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56829b6f-bf72-4de7-b507-5563e461811d None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:22.621980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56829b6f-bf72-4de7-b507-5563e461811d None admin] PATH_INFO: `/v3/projects/service` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:22.628585 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56829b6f-bf72-4de7-b507-5563e461811d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:22.633663 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56829b6f-bf72-4de7-b507-5563e461811d None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:22.640966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-56829b6f-bf72-4de7-b507-5563e461811d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:48:22.640966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:22.640966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:22.640966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:22.640966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:22.640966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:22.640966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:22.640966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:22.640966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:22.640966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:22.640966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:22.640966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:22.640966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:48:22.640966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:48:22.640966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:48:22.641852 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:48:22.641852 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:22.641852 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:22.641852 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:22.641852 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:22.641852 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:22.641852 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:22.641852 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:22.641852 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:22.641852 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:22.641852 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:22.641852 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:22.641852 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:22.641852 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:22.641852 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:22.641852 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:22.641852 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:22.642822 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:48:22.642822 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:48:22.642822 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:48:22.642822 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:48:22.642822 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:48:22.642822 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:48:22.642822 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:48:22.642822 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:48:22.642822 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 631/1258] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:48:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:22.647922 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68cfb1e-3c13-4ddb-ae95-0184b13adecd None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:22.747642 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68cfb1e-3c13-4ddb-ae95-0184b13adecd None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:22.781395 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb731e6-e76a-4446-893d-37a334431ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:22.782032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb731e6-e76a-4446-893d-37a334431ee8 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:22.782304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb731e6-e76a-4446-893d-37a334431ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:22.782602 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb731e6-e76a-4446-893d-37a334431ee8 None admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:22.783290 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb731e6-e76a-4446-893d-37a334431ee8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:22.783595 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb731e6-e76a-4446-893d-37a334431ee8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:22.785021 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb731e6-e76a-4446-893d-37a334431ee8 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:22.798752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 628/1259] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:48:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:22.804868 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56829b6f-bf72-4de7-b507-5563e461811d None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:22.829715 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:22.903713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56829b6f-bf72-4de7-b507-5563e461811d None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:22.938665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd01612-06ca-476c-8227-75383071aae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:22.939266 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd01612-06ca-476c-8227-75383071aae0 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:22.939543 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd01612-06ca-476c-8227-75383071aae0 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:22.939812 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd01612-06ca-476c-8227-75383071aae0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:22.940578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd01612-06ca-476c-8227-75383071aae0 None admin] RBAC: Adding query filter params (role.id=13ca33d694c645d6920930d39dc52a0d, scope.project.id=8db9b3f8a9f041938ac7b39003a35015, user.id=f3db46a4b4e74cc0bb2a7bd6f0e99105) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:22.940886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd01612-06ca-476c-8227-75383071aae0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:22.942305 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd01612-06ca-476c-8227-75383071aae0 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:22.947700 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 632/1260] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:48:22 2022] GET /identity/v3/role_assignments?user.id=f3db46a4b4e74cc0bb2a7bd6f0e99105&scope.project.id=8db9b3f8a9f041938ac7b39003a35015&role.id=13ca33d694c645d6920930d39dc52a0d => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:24.812907 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf1d00-4c33-41f1-863b-493e755236d5 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:24.813477 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf1d00-4c33-41f1-863b-493e755236d5 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:24.813477 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf1d00-4c33-41f1-863b-493e755236d5 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:24.814902 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 629/1261] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:48:24.820004 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68a0bb-6551-4150-bdc2-7e23994c9958 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:24.820380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68a0bb-6551-4150-bdc2-7e23994c9958 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:24.820543 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68a0bb-6551-4150-bdc2-7e23994c9958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:24.839212 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-3a68a0bb-6551-4150-bdc2-7e23994c9958 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:48:24.840380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-3a68a0bb-6551-4150-bdc2-7e23994c9958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:48:24.907467 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 633/1262] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:48:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:48:25.178709 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf1d00-4c33-41f1-863b-493e755236d5 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:25.190346 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:25.280261 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf1d00-4c33-41f1-863b-493e755236d5 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:25.314415 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a6b6c-2085-4678-b266-6f6a02b77ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:25.315036 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a6b6c-2085-4678-b266-6f6a02b77ee5 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:25.315310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a6b6c-2085-4678-b266-6f6a02b77ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:25.315582 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a6b6c-2085-4678-b266-6f6a02b77ee5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:25.325595 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8a6b6c-2085-4678-b266-6f6a02b77ee5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:48:25.330303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-1e8a6b6c-2085-4678-b266-6f6a02b77ee5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:48:25.330303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:25.330303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:25.330303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:25.330303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:25.330303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:25.330303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:25.330303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:25.330303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:25.330303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:25.330303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:25.330303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:25.330303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:48:25.330303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:48:25.330303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:48:25.330303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:48:25.331475 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:48:25.331475 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:48:25.331475 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:48:25.331475 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:48:25.331475 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:25.331475 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:25.331475 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:25.331475 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:25.331475 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:25.331475 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:25.331475 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:25.331475 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:25.331475 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:25.331475 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:25.331475 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:25.331475 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:25.332367 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:25.332367 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:25.332367 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:25.332367 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:25.332367 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:48:25.332367 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:48:25.332367 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:48:25.332367 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:48:25.332367 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:48:25.332367 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:48:25.332367 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:48:25.332367 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:48:25.332367 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 630/1263] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:48:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:25.337658 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68a0bb-6551-4150-bdc2-7e23994c9958 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:25.355864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:25.439148 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68a0bb-6551-4150-bdc2-7e23994c9958 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:25.472853 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb3daad-080e-468b-a7b5-84f8f266f79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:25.473454 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3daad-080e-468b-a7b5-84f8f266f79f None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:25.473728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3daad-080e-468b-a7b5-84f8f266f79f None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:25.474002 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3daad-080e-468b-a7b5-84f8f266f79f None admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:25.474681 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb3daad-080e-468b-a7b5-84f8f266f79f None admin] RBAC: Adding query filter params (name=admin) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:25.474978 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb3daad-080e-468b-a7b5-84f8f266f79f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:25.476329 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb3daad-080e-468b-a7b5-84f8f266f79f None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:25.484672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 634/1264] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:48:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:25.490946 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a6b6c-2085-4678-b266-6f6a02b77ee5 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:25.559253 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:25.592500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a6b6c-2085-4678-b266-6f6a02b77ee5 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:25.627015 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf199684-71e7-4969-9bff-93eb1741bc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:25.627637 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf199684-71e7-4969-9bff-93eb1741bc50 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:25.627914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf199684-71e7-4969-9bff-93eb1741bc50 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:25.628184 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf199684-71e7-4969-9bff-93eb1741bc50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:25.637924 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf199684-71e7-4969-9bff-93eb1741bc50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:25.639653 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf199684-71e7-4969-9bff-93eb1741bc50 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:25.656461 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-cf199684-71e7-4969-9bff-93eb1741bc50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:48:25.656461 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:25.656461 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:25.656461 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:25.656461 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:25.656461 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:25.656461 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:25.656461 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:25.656461 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:25.656461 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:25.656461 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:25.656461 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:25.656461 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:48:25.656461 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:48:25.656461 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:48:25.656461 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:48:25.657617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:25.657617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:25.657617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:25.657617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:25.657617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:25.657617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:25.657617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:25.657617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:25.657617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:25.657617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:25.657617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:25.657617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:25.657617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:25.657617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:25.657617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:25.657617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:25.657617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:48:25.658707 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:48:25.658707 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:48:25.658707 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:48:25.658707 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 631/1265] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:48:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:25.663729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb3daad-080e-468b-a7b5-84f8f266f79f None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:25.746032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:25.764740 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb3daad-080e-468b-a7b5-84f8f266f79f None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:25.797991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f383b-d075-4654-a45d-4b3ba819514e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:25.798686 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f383b-d075-4654-a45d-4b3ba819514e None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:25.798963 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f383b-d075-4654-a45d-4b3ba819514e None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:25.799231 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f383b-d075-4654-a45d-4b3ba819514e None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:25.799860 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f383b-d075-4654-a45d-4b3ba819514e None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:25.800159 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f383b-d075-4654-a45d-4b3ba819514e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:25.801550 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f383b-d075-4654-a45d-4b3ba819514e None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:25.812323 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 635/1266] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:25.818094 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf199684-71e7-4969-9bff-93eb1741bc50 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:25.919726 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf199684-71e7-4969-9bff-93eb1741bc50 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:25.939549 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:25.952988 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d8ae4-4ca7-4fbd-a60a-d0301f814b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:25.953610 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d8ae4-4ca7-4fbd-a60a-d0301f814b8f None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:25.953884 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d8ae4-4ca7-4fbd-a60a-d0301f814b8f None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:25.954151 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d8ae4-4ca7-4fbd-a60a-d0301f814b8f None admin] PATH_INFO: `/v3/users/placement` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:25.961361 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d8ae4-4ca7-4fbd-a60a-d0301f814b8f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:25.962817 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d8ae4-4ca7-4fbd-a60a-d0301f814b8f None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:25.973763 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-355d8ae4-4ca7-4fbd-a60a-d0301f814b8f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 10:48:25.973763 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:25.973763 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:25.973763 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:25.973763 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:25.973763 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:25.973763 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:25.973763 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:25.973763 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:25.973763 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:25.973763 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:25.973763 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:25.973763 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:48:25.973763 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:48:25.973763 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:48:25.973763 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:48:25.982138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:25.982138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:25.982138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:48:25.982138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:48:25.982138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:48:25.982138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:48:25.982138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:25.982138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:25.982138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:25.982138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:25.982138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:25.982138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:25.982138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:25.982138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:25.982138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:25.982138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:25.982138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:25.982138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:25.983341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:25.983341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:25.983341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:48:25.983341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:48:25.983341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:48:25.983341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:48:25.983341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:48:25.983341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:48:25.983341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 10:48:25.983341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:48:25.983341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 632/1267] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:48:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:25.984211 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f383b-d075-4654-a45d-4b3ba819514e None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:26.082668 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f383b-d075-4654-a45d-4b3ba819514e None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:26.116194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71655d9-f6dc-4655-a7a4-b9d842638d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:26.116793 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71655d9-f6dc-4655-a7a4-b9d842638d0e None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:26.117376 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71655d9-f6dc-4655-a7a4-b9d842638d0e None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:26.117638 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71655d9-f6dc-4655-a7a4-b9d842638d0e None admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:26.118355 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71655d9-f6dc-4655-a7a4-b9d842638d0e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:26.118683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71655d9-f6dc-4655-a7a4-b9d842638d0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:26.120097 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71655d9-f6dc-4655-a7a4-b9d842638d0e None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:26.130091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:26.138015 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 636/1268] 10.222.0.16 () {56 vars in 1143 bytes} [Wed Aug 31 10:48:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:26.143796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d8ae4-4ca7-4fbd-a60a-d0301f814b8f None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:26.242262 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d8ae4-4ca7-4fbd-a60a-d0301f814b8f None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:26.274719 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7a25a-5e40-4924-8e2f-d0699afa3e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:26.275332 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7a25a-5e40-4924-8e2f-d0699afa3e26 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:26.275604 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7a25a-5e40-4924-8e2f-d0699afa3e26 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:26.275869 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7a25a-5e40-4924-8e2f-d0699afa3e26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:26.286582 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b7a25a-5e40-4924-8e2f-d0699afa3e26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:26.288012 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b7a25a-5e40-4924-8e2f-d0699afa3e26 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:26.302569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-b4b7a25a-5e40-4924-8e2f-d0699afa3e26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:48:26.302569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:26.302569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:26.302569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:26.302569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:26.302569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:26.302569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:26.302569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:26.302569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:26.302569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:26.302569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:26.302569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:26.302569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:48:26.302569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:48:26.302569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:48:26.302569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:48:26.303660 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:26.303660 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:26.303660 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:26.303660 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:26.303660 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:26.303660 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:26.303660 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:26.303660 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:26.303660 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:26.303660 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:26.303660 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:26.303660 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:26.303660 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:26.303660 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:26.303660 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:26.303660 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:26.303660 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:48:26.304808 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:48:26.304808 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:48:26.304808 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:48:26.304808 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 633/1269] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:48:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:26.310368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71655d9-f6dc-4655-a7a4-b9d842638d0e None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:26.411005 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71655d9-f6dc-4655-a7a4-b9d842638d0e None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:26.446376 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2b51f-d312-4e94-984f-28fce8b4c159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:26.447000 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2b51f-d312-4e94-984f-28fce8b4c159 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:26.447277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2b51f-d312-4e94-984f-28fce8b4c159 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:26.447548 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2b51f-d312-4e94-984f-28fce8b4c159 None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:26.448180 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b2b51f-d312-4e94-984f-28fce8b4c159 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:26.448481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b2b51f-d312-4e94-984f-28fce8b4c159 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:26.449933 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b2b51f-d312-4e94-984f-28fce8b4c159 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:26.461657 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 637/1270] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:48:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:26.467552 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7a25a-5e40-4924-8e2f-d0699afa3e26 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:26.499630 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:26.568766 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7a25a-5e40-4924-8e2f-d0699afa3e26 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:26.601599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a6c21-006d-4b86-b7b4-fdf67a0d3180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:26.602246 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a6c21-006d-4b86-b7b4-fdf67a0d3180 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:26.602550 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a6c21-006d-4b86-b7b4-fdf67a0d3180 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:26.602815 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a6c21-006d-4b86-b7b4-fdf67a0d3180 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:26.609599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a6c21-006d-4b86-b7b4-fdf67a0d3180 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:26.611130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a6c21-006d-4b86-b7b4-fdf67a0d3180 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:26.622973 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-e20a6c21-006d-4b86-b7b4-fdf67a0d3180 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:48:26.622973 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:26.622973 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:26.622973 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:26.622973 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:26.622973 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:26.622973 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:26.622973 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:26.622973 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:26.622973 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:26.622973 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:26.622973 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:26.622973 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:48:26.622973 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:48:26.622973 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:48:26.624068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:48:26.624068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:26.624068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:26.624068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:26.624068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:26.624068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:26.624068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:26.624068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:26.624068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:26.624068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:26.624068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:26.624068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:26.624068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:26.624068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:26.624068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:26.624068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:26.624068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:26.625136 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:48:26.625136 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:48:26.625136 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:48:26.625136 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:48:26.625136 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:48:26.625136 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:48:26.625136 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:48:26.625136 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:48:26.625136 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 634/1271] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:48:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:26.630028 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2b51f-d312-4e94-984f-28fce8b4c159 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:26.691837 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:26.732980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2b51f-d312-4e94-984f-28fce8b4c159 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:26.765840 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47501f-d6bc-4d91-9a08-d477d25adec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:26.766499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47501f-d6bc-4d91-9a08-d477d25adec9 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:26.766774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47501f-d6bc-4d91-9a08-d477d25adec9 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:26.767041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47501f-d6bc-4d91-9a08-d477d25adec9 None admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:26.767719 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a47501f-d6bc-4d91-9a08-d477d25adec9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:26.768018 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a47501f-d6bc-4d91-9a08-d477d25adec9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:26.769432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a47501f-d6bc-4d91-9a08-d477d25adec9 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:26.781139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 638/1272] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:48:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:26.787210 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a6c21-006d-4b86-b7b4-fdf67a0d3180 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:26.878280 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:26.886722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a6c21-006d-4b86-b7b4-fdf67a0d3180 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:26.920799 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42107ff5-ddae-469c-9e72-6f7483695a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:26.921448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42107ff5-ddae-469c-9e72-6f7483695a4c None admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:26.921723 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42107ff5-ddae-469c-9e72-6f7483695a4c None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:26.921992 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42107ff5-ddae-469c-9e72-6f7483695a4c None admin] PATH_INFO: `/v3/projects/8db9b3f8a9f041938ac7b39003a35015/users/f3db46a4b4e74cc0bb2a7bd6f0e99105/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:26.923941 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42107ff5-ddae-469c-9e72-6f7483695a4c None admin] RBAC: Authorizing `identity:create_grant(project_id=8db9b3f8a9f041938ac7b39003a35015, user_id=f3db46a4b4e74cc0bb2a7bd6f0e99105, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:26.925388 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42107ff5-ddae-469c-9e72-6f7483695a4c None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:27.005643 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 635/1273] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 31 10:48:26 2022] PUT /identity/v3/projects/8db9b3f8a9f041938ac7b39003a35015/users/f3db46a4b4e74cc0bb2a7bd6f0e99105/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:48:28.915802 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae8ed4-ead6-4aa1-bf16-5644212a4a88 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:28.916349 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae8ed4-ead6-4aa1-bf16-5644212a4a88 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:28.916349 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae8ed4-ead6-4aa1-bf16-5644212a4a88 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:28.917765 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 639/1274] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:48:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:48:28.923681 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46f70a-1dd2-45cb-b955-14b1c491f97c None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:28.924097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46f70a-1dd2-45cb-b955-14b1c491f97c None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:28.924224 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46f70a-1dd2-45cb-b955-14b1c491f97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:28.943176 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-2c46f70a-1dd2-45cb-b955-14b1c491f97c None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:48:28.944351 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-2c46f70a-1dd2-45cb-b955-14b1c491f97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:48:29.009235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 636/1275] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:48:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:48:29.283702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae8ed4-ead6-4aa1-bf16-5644212a4a88 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:29.386369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae8ed4-ead6-4aa1-bf16-5644212a4a88 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:29.403815 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:29.419883 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053a496-fdc4-486b-8598-eb6275bd3958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:29.420495 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053a496-fdc4-486b-8598-eb6275bd3958 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:29.420766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053a496-fdc4-486b-8598-eb6275bd3958 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:29.421064 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053a496-fdc4-486b-8598-eb6275bd3958 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:29.430684 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2053a496-fdc4-486b-8598-eb6275bd3958 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:48:29.436097 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-2053a496-fdc4-486b-8598-eb6275bd3958 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:48:29.436097 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:29.436097 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:29.436097 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:29.436097 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:29.436097 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:29.436097 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:29.436097 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:29.436097 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:29.436097 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:29.436097 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:29.436097 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:29.436097 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:48:29.436097 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:48:29.436097 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:48:29.436097 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:48:29.437241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:48:29.437241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:48:29.437241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:48:29.437241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:48:29.437241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:29.437241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:29.437241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:29.437241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:29.437241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:29.437241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:29.437241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:29.437241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:29.437241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:29.437241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:29.437241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:29.437241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:29.438283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:29.438283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:29.438283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:29.438283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:29.438283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:48:29.438283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:48:29.438283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:48:29.438283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:48:29.438283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:48:29.438283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:48:29.438283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:48:29.438283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:48:29.438283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 640/1276] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 31 10:48:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:29.443156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46f70a-1dd2-45cb-b955-14b1c491f97c None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:29.523675 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:29.540796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46f70a-1dd2-45cb-b955-14b1c491f97c None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:29.573703 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d785306b-06f2-4cd7-bdfd-4d25c5d7da1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:29.574302 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785306b-06f2-4cd7-bdfd-4d25c5d7da1d None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:29.574603 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785306b-06f2-4cd7-bdfd-4d25c5d7da1d None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:29.574868 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785306b-06f2-4cd7-bdfd-4d25c5d7da1d None admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:29.576199 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d785306b-06f2-4cd7-bdfd-4d25c5d7da1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:29.576701 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d785306b-06f2-4cd7-bdfd-4d25c5d7da1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:29.578356 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d785306b-06f2-4cd7-bdfd-4d25c5d7da1d None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:29.586410 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 637/1277] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 31 10:48:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:29.592535 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053a496-fdc4-486b-8598-eb6275bd3958 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:29.692448 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053a496-fdc4-486b-8598-eb6275bd3958 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:29.725893 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dbeb98-eb90-4ebc-9775-99bf14fa4cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:29.726530 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dbeb98-eb90-4ebc-9775-99bf14fa4cf9 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:29.726806 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dbeb98-eb90-4ebc-9775-99bf14fa4cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:29.727074 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dbeb98-eb90-4ebc-9775-99bf14fa4cf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:29.737510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dbeb98-eb90-4ebc-9775-99bf14fa4cf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:29.738987 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dbeb98-eb90-4ebc-9775-99bf14fa4cf9 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:29.754462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-d6dbeb98-eb90-4ebc-9775-99bf14fa4cf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:48:29.754462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:29.754462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:29.754462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:29.754462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:29.754462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:29.754462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:29.754462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:29.754462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:29.754462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:29.754462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:29.754462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:29.754462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:48:29.754462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:48:29.754462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:48:29.754462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:48:29.755562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:29.755562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:29.755562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:29.755562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:29.755562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:29.755562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:29.755562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:29.755562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:29.755562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:29.755562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:29.755562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:29.755562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:29.755562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:29.755562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:29.755562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:29.755562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:29.755562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:48:29.756648 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:48:29.756648 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:48:29.756648 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:48:29.756648 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 641/1278] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:48:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:29.761687 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d785306b-06f2-4cd7-bdfd-4d25c5d7da1d None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:29.860908 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d785306b-06f2-4cd7-bdfd-4d25c5d7da1d None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:29.894201 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd2c84-f52d-462c-9420-9ae9c532dd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:29.894828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd2c84-f52d-462c-9420-9ae9c532dd0b None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:29.895103 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd2c84-f52d-462c-9420-9ae9c532dd0b None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:29.895371 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd2c84-f52d-462c-9420-9ae9c532dd0b None admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:29.896003 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fd2c84-f52d-462c-9420-9ae9c532dd0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:29.896299 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fd2c84-f52d-462c-9420-9ae9c532dd0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:29.897648 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fd2c84-f52d-462c-9420-9ae9c532dd0b None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:29.909448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 638/1279] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:48:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:29.915301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dbeb98-eb90-4ebc-9775-99bf14fa4cf9 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:29.948248 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:30.022004 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dbeb98-eb90-4ebc-9775-99bf14fa4cf9 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:30.056422 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628087eb-8d6d-408a-a700-f0bd973ce1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:30.057037 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628087eb-8d6d-408a-a700-f0bd973ce1da None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:30.057310 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628087eb-8d6d-408a-a700-f0bd973ce1da None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:30.057576 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628087eb-8d6d-408a-a700-f0bd973ce1da None admin] PATH_INFO: `/v3/users/placement` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:30.064971 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628087eb-8d6d-408a-a700-f0bd973ce1da None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:30.066416 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628087eb-8d6d-408a-a700-f0bd973ce1da None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:30.078238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-628087eb-8d6d-408a-a700-f0bd973ce1da None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 10:48:30.078238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:30.078238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:30.078238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:30.078238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:30.078238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:30.078238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:30.078238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:30.078238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:30.078238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:30.078238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:30.078238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:30.078238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:48:30.078238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:48:30.078238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:48:30.078238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:48:30.079374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:30.079374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:30.079374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:48:30.079374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:48:30.079374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:48:30.079374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:48:30.079374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:30.079374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:30.079374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:30.079374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:30.079374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:30.079374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:30.079374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:30.079374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:30.079374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:30.079374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:30.079374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:30.079374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:30.080504 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:30.080504 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:30.080504 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:48:30.080504 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:48:30.080504 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:48:30.080504 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:48:30.080504 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:48:30.080504 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:48:30.080504 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 10:48:30.080504 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:48:30.080504 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 642/1280] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:48:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:30.085348 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd2c84-f52d-462c-9420-9ae9c532dd0b None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:30.091311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:30.183798 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd2c84-f52d-462c-9420-9ae9c532dd0b None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:30.218549 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f88df-6591-4e1b-be46-9cb0ebc53149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:30.219160 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f88df-6591-4e1b-be46-9cb0ebc53149 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:30.219438 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f88df-6591-4e1b-be46-9cb0ebc53149 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:30.219726 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f88df-6591-4e1b-be46-9cb0ebc53149 None admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:30.220435 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f88df-6591-4e1b-be46-9cb0ebc53149 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:30.220738 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f88df-6591-4e1b-be46-9cb0ebc53149 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:30.222277 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f88df-6591-4e1b-be46-9cb0ebc53149 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:30.239600 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 639/1281] 10.222.0.16 () {56 vars in 1143 bytes} [Wed Aug 31 10:48:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:48:30.245816 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628087eb-8d6d-408a-a700-f0bd973ce1da None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:30.334793 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:30.343914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628087eb-8d6d-408a-a700-f0bd973ce1da None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:30.378303 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6ecc5-3f2c-4a47-a26b-3564d28d672a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:30.378930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6ecc5-3f2c-4a47-a26b-3564d28d672a None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:30.379208 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6ecc5-3f2c-4a47-a26b-3564d28d672a None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:30.379472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6ecc5-3f2c-4a47-a26b-3564d28d672a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:30.446246 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d6ecc5-3f2c-4a47-a26b-3564d28d672a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:30.447637 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d6ecc5-3f2c-4a47-a26b-3564d28d672a None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:30.534369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:30.544979 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 643/1282] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 31 10:48:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:48:30.550960 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f88df-6591-4e1b-be46-9cb0ebc53149 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:30.613076 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:30.651881 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f88df-6591-4e1b-be46-9cb0ebc53149 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:30.687421 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392bbc4-8ff8-410b-b314-1771d6d025a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:30.688044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392bbc4-8ff8-410b-b314-1771d6d025a5 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:30.688319 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392bbc4-8ff8-410b-b314-1771d6d025a5 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:30.688587 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392bbc4-8ff8-410b-b314-1771d6d025a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:30.698616 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0392bbc4-8ff8-410b-b314-1771d6d025a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:30.699998 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0392bbc4-8ff8-410b-b314-1771d6d025a5 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:30.718173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-0392bbc4-8ff8-410b-b314-1771d6d025a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:48:30.718173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:30.718173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:30.718173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:30.718173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:30.718173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:30.718173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:30.718173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:30.718173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:30.718173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:30.718173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:30.718173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:30.718173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:48:30.718173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:48:30.718173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:48:30.718173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:48:30.720640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:30.720640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:30.720640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:30.720640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:30.720640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:30.720640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:30.720640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:30.720640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:30.720640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:30.720640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:30.720640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:30.720640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:30.720640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:30.720640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:30.720640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:30.720640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:30.720640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:48:30.721521 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:48:30.721521 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:48:30.721521 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:48:30.721521 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 640/1283] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:48:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:30.725389 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6ecc5-3f2c-4a47-a26b-3564d28d672a None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:30.823908 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6ecc5-3f2c-4a47-a26b-3564d28d672a None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:30.858075 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87508581-cf14-4e35-94d4-279b34032dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:30.858728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87508581-cf14-4e35-94d4-279b34032dc6 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:30.859002 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87508581-cf14-4e35-94d4-279b34032dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:30.859271 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87508581-cf14-4e35-94d4-279b34032dc6 None admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:30.859897 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87508581-cf14-4e35-94d4-279b34032dc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:30.860192 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87508581-cf14-4e35-94d4-279b34032dc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:30.861626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87508581-cf14-4e35-94d4-279b34032dc6 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:30.873742 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 644/1284] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 31 10:48:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:30.879423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392bbc4-8ff8-410b-b314-1771d6d025a5 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:30.976295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392bbc4-8ff8-410b-b314-1771d6d025a5 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:30.989275 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:31.008917 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cf155-cebb-4eb1-9938-cfb1beb0e16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:31.009526 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cf155-cebb-4eb1-9938-cfb1beb0e16f None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:31.009801 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cf155-cebb-4eb1-9938-cfb1beb0e16f None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:31.010069 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cf155-cebb-4eb1-9938-cfb1beb0e16f None admin] PATH_INFO: `/v3/projects/service` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:31.017127 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4cf155-cebb-4eb1-9938-cfb1beb0e16f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:31.018589 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4cf155-cebb-4eb1-9938-cfb1beb0e16f None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:31.029428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application [None req-0d4cf155-cebb-4eb1-9938-cfb1beb0e16f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:48:31.029428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:31.029428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:31.029428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:31.029428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:31.029428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:31.029428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:31.029428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:31.029428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:31.029428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:31.029428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:31.029428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:31.029428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:48:31.029428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:48:31.029428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:48:31.030532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:48:31.030532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:31.030532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:31.030532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:31.030532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:31.030532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:31.030532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:31.030532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:31.030532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:31.030532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:31.030532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:31.030532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:31.030532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:31.030532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:31.030532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:31.030532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:31.030532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:31.031646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:48:31.031646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:48:31.031646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:48:31.031646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:48:31.031646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:48:31.031646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:48:31.031646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:48:31.031646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: ERROR keystone.server.flask.application Aug 31 10:48:31.031646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 641/1285] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 31 10:48:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:31.036354 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87508581-cf14-4e35-94d4-279b34032dc6 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:31.086988 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:31.136021 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87508581-cf14-4e35-94d4-279b34032dc6 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:31.168485 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9eaa2c-be33-448e-9c3d-c1f9e9206e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:31.169588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9eaa2c-be33-448e-9c3d-c1f9e9206e04 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:31.169860 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9eaa2c-be33-448e-9c3d-c1f9e9206e04 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:31.170127 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9eaa2c-be33-448e-9c3d-c1f9e9206e04 None admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:31.170835 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9eaa2c-be33-448e-9c3d-c1f9e9206e04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:31.171136 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9eaa2c-be33-448e-9c3d-c1f9e9206e04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:31.172545 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9eaa2c-be33-448e-9c3d-c1f9e9206e04 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:31.183921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 645/1286] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 31 10:48:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:31.190695 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cf155-cebb-4eb1-9938-cfb1beb0e16f None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:31.289602 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cf155-cebb-4eb1-9938-cfb1beb0e16f None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:31.321929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db591a7-df36-4314-b7ed-e55ff97d8524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:31.322566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db591a7-df36-4314-b7ed-e55ff97d8524 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:31.322846 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db591a7-df36-4314-b7ed-e55ff97d8524 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:31.323115 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db591a7-df36-4314-b7ed-e55ff97d8524 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:31.323830 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db591a7-df36-4314-b7ed-e55ff97d8524 None admin] RBAC: Adding query filter params (role.id=13ca33d694c645d6920930d39dc52a0d, scope.project.id=8db9b3f8a9f041938ac7b39003a35015, user.id=f3db46a4b4e74cc0bb2a7bd6f0e99105) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:31.324131 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db591a7-df36-4314-b7ed-e55ff97d8524 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:31.326151 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db591a7-df36-4314-b7ed-e55ff97d8524 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:31.331923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 642/1287] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 31 10:48:31 2022] GET /identity/v3/role_assignments?user.id=f3db46a4b4e74cc0bb2a7bd6f0e99105&scope.project.id=8db9b3f8a9f041938ac7b39003a35015&role.id=13ca33d694c645d6920930d39dc52a0d => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:33.301911 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b4c28-3570-48e2-a54b-ec88c371f5c3 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:33.302555 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b4c28-3570-48e2-a54b-ec88c371f5c3 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:33.302555 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b4c28-3570-48e2-a54b-ec88c371f5c3 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:33.303816 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 646/1288] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:48:33.308910 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f421e57-01e5-4b7f-89f3-0ae0c4f6ef3c None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:33.309331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f421e57-01e5-4b7f-89f3-0ae0c4f6ef3c None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:33.309999 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f421e57-01e5-4b7f-89f3-0ae0c4f6ef3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:33.328625 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-0f421e57-01e5-4b7f-89f3-0ae0c4f6ef3c None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:48:33.329809 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-0f421e57-01e5-4b7f-89f3-0ae0c4f6ef3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:48:33.359572 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:33.396700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 643/1289] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:48:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:48:33.664583 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b4c28-3570-48e2-a54b-ec88c371f5c3 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:33.768436 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b4c28-3570-48e2-a54b-ec88c371f5c3 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:33.779982 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:33.803691 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881670f-3643-4cd5-8aad-8df02a2557ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:33.804504 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881670f-3643-4cd5-8aad-8df02a2557ed None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:33.804780 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881670f-3643-4cd5-8aad-8df02a2557ed None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:33.805048 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881670f-3643-4cd5-8aad-8df02a2557ed None admin] PATH_INFO: `/v3/services/placement` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:33.805626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d881670f-3643-4cd5-8aad-8df02a2557ed None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:33.807014 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d881670f-3643-4cd5-8aad-8df02a2557ed None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:33.816481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-d881670f-3643-4cd5-8aad-8df02a2557ed None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 10:48:33.816481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:33.816481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:48:33.816481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:48:33.816481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:48:33.816481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:48:33.816481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:48:33.816481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:48:33.816481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 10:48:33.816481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:48:33.816481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:48:33.816481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:48:33.816481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:33.816481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:33.816481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:33.816481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:33.816481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:33.817851 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:33.817851 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:33.817851 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:33.817851 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:33.817851 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:33.817851 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:33.817851 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:48:33.817851 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:48:33.817851 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:48:33.817851 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:48:33.817851 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:33.817851 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:33.817851 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:33.817851 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:33.817851 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:33.817851 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:33.819095 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:33.819095 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:33.819095 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:33.819095 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:33.819095 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:33.819095 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:33.819095 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:33.819095 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:33.819095 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:33.819095 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:33.819095 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:48:33.819095 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:48:33.819095 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 10:48:33.819095 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:48:33.819095 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 647/1290] 10.222.0.16 () {56 vars in 1117 bytes} [Wed Aug 31 10:48:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:48:33.823671 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f421e57-01e5-4b7f-89f3-0ae0c4f6ef3c None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:33.932848 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f421e57-01e5-4b7f-89f3-0ae0c4f6ef3c None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:33.972799 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae94f06-572d-4a87-af1b-3f50487d79c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:33.973503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae94f06-572d-4a87-af1b-3f50487d79c1 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:33.973783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae94f06-572d-4a87-af1b-3f50487d79c1 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:33.974100 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae94f06-572d-4a87-af1b-3f50487d79c1 None admin] PATH_INFO: `/v3/services` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:33.974789 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae94f06-572d-4a87-af1b-3f50487d79c1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:33.975107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae94f06-572d-4a87-af1b-3f50487d79c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:33.976564 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae94f06-572d-4a87-af1b-3f50487d79c1 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:33.987062 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 644/1291] 10.222.0.16 () {56 vars in 1116 bytes} [Wed Aug 31 10:48:33 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:33.988326 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881670f-3643-4cd5-8aad-8df02a2557ed None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:34.100783 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881670f-3643-4cd5-8aad-8df02a2557ed None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:34.139122 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9653c82-9a7d-4804-8f60-da2e3e975486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:34.139744 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9653c82-9a7d-4804-8f60-da2e3e975486 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:34.140024 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9653c82-9a7d-4804-8f60-da2e3e975486 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:34.140294 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9653c82-9a7d-4804-8f60-da2e3e975486 None admin] PATH_INFO: `/v3/services` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:34.140996 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9653c82-9a7d-4804-8f60-da2e3e975486 None admin] RBAC: Adding query filter params (type=placement) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:34.141306 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9653c82-9a7d-4804-8f60-da2e3e975486 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:34.142727 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9653c82-9a7d-4804-8f60-da2e3e975486 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:34.147865 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 648/1292] 10.222.0.16 () {56 vars in 1116 bytes} [Wed Aug 31 10:48:33 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:36.050366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ad518-3917-4d89-99d1-0867a134941d None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:36.050952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ad518-3917-4d89-99d1-0867a134941d None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:36.050952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ad518-3917-4d89-99d1-0867a134941d None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:36.052282 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 645/1293] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:48:36.057349 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c297b-6584-40f2-ae9e-5a10e379ff99 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:36.057736 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c297b-6584-40f2-ae9e-5a10e379ff99 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:36.057888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c297b-6584-40f2-ae9e-5a10e379ff99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:36.077333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-4f4c297b-6584-40f2-ae9e-5a10e379ff99 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:48:36.078568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-4f4c297b-6584-40f2-ae9e-5a10e379ff99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:48:36.108704 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:36.144994 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 649/1294] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:48:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:48:36.411242 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ad518-3917-4d89-99d1-0867a134941d None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:36.442948 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:36.515574 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ad518-3917-4d89-99d1-0867a134941d None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:36.550010 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d538e58-9d75-464d-bdc9-dbe1e09f4a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:36.550632 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d538e58-9d75-464d-bdc9-dbe1e09f4a9a None admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:36.550906 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d538e58-9d75-464d-bdc9-dbe1e09f4a9a None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:36.551174 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d538e58-9d75-464d-bdc9-dbe1e09f4a9a None admin] PATH_INFO: `/v3/services` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:36.551821 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d538e58-9d75-464d-bdc9-dbe1e09f4a9a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:36.553168 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d538e58-9d75-464d-bdc9-dbe1e09f4a9a None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:36.640416 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 646/1295] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Aug 31 10:48:36 2022] POST /identity/v3/services => generated 246 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:48:38.658779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2023c-8fac-4480-abb6-d3972113007a None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:38.659341 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2023c-8fac-4480-abb6-d3972113007a None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:38.659493 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2023c-8fac-4480-abb6-d3972113007a None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:38.660887 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 650/1296] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:48:38.666029 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734b4a7-43a6-475a-ab57-746e310e35a0 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:38.666512 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734b4a7-43a6-475a-ab57-746e310e35a0 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:38.666616 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734b4a7-43a6-475a-ab57-746e310e35a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:38.687110 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-0734b4a7-43a6-475a-ab57-746e310e35a0 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:48:38.688333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-0734b4a7-43a6-475a-ab57-746e310e35a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:48:38.769694 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 647/1297] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:48:38 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:48:39.072804 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f2023c-8fac-4480-abb6-d3972113007a None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:39.185895 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f2023c-8fac-4480-abb6-d3972113007a None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:39.220787 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a608d9-f146-40e2-8b55-4edb4cba9de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:39.224036 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a608d9-f146-40e2-8b55-4edb4cba9de5 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:39.224036 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a608d9-f146-40e2-8b55-4edb4cba9de5 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:39.224036 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a608d9-f146-40e2-8b55-4edb4cba9de5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:39.224036 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a608d9-f146-40e2-8b55-4edb4cba9de5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:39.224036 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a608d9-f146-40e2-8b55-4edb4cba9de5 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:39.233217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-95a608d9-f146-40e2-8b55-4edb4cba9de5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 10:48:39.233217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:39.233217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:48:39.233217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:48:39.233217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:48:39.233217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:48:39.233217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:48:39.233217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:48:39.233217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 10:48:39.233217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:48:39.233217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:48:39.233217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:48:39.233217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:39.233217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:39.233217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:39.233217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:39.233217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:39.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:39.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:39.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:39.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:39.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:39.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:39.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:48:39.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:48:39.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:48:39.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:48:39.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:39.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:39.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:39.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:39.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:39.234242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:39.235196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:39.235196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:39.235196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:39.235196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:39.235196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:39.235196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:39.235196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:39.235196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:39.235196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:39.235196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:39.235196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:48:39.235196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:48:39.235196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 10:48:39.235196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:48:39.235196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 651/1298] 10.222.0.16 () {56 vars in 1117 bytes} [Wed Aug 31 10:48:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:39.240320 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734b4a7-43a6-475a-ab57-746e310e35a0 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:39.258577 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:39.367510 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734b4a7-43a6-475a-ab57-746e310e35a0 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:39.401711 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5ab27-0754-4a68-9496-16bb798839e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:39.402308 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5ab27-0754-4a68-9496-16bb798839e2 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:39.402609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5ab27-0754-4a68-9496-16bb798839e2 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:39.402878 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5ab27-0754-4a68-9496-16bb798839e2 None admin] PATH_INFO: `/v3/services` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:39.403639 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5ab27-0754-4a68-9496-16bb798839e2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:39.403922 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5ab27-0754-4a68-9496-16bb798839e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:39.405341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5ab27-0754-4a68-9496-16bb798839e2 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:39.410476 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 648/1299] 10.222.0.16 () {56 vars in 1116 bytes} [Wed Aug 31 10:48:39 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:39.416586 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a608d9-f146-40e2-8b55-4edb4cba9de5 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:39.448803 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:39.517153 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a608d9-f146-40e2-8b55-4edb4cba9de5 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:39.550658 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4342a5-fcf0-4260-82de-1a490f4590b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:39.551262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4342a5-fcf0-4260-82de-1a490f4590b7 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:39.551538 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4342a5-fcf0-4260-82de-1a490f4590b7 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:39.551807 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4342a5-fcf0-4260-82de-1a490f4590b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:39.552501 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4342a5-fcf0-4260-82de-1a490f4590b7 None admin] RBAC: Adding query filter params (interface=public, service_id=b1c44e5ef64f4532be6c2f9e7ccc9f9b, region_id=RegionOne) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:39.552805 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4342a5-fcf0-4260-82de-1a490f4590b7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:39.554160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4342a5-fcf0-4260-82de-1a490f4590b7 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:39.559146 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 652/1300] 10.222.0.16 () {56 vars in 1251 bytes} [Wed Aug 31 10:48:39 2022] GET /identity/v3/endpoints?service_id=b1c44e5ef64f4532be6c2f9e7ccc9f9b&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:39.565538 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5ab27-0754-4a68-9496-16bb798839e2 None admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:39.648483 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:39.666060 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5ab27-0754-4a68-9496-16bb798839e2 None admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:39.700699 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6610de3-3452-4fb1-90b8-4ca1a06b3665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:39.701298 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6610de3-3452-4fb1-90b8-4ca1a06b3665 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:39.701568 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6610de3-3452-4fb1-90b8-4ca1a06b3665 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:39.701832 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6610de3-3452-4fb1-90b8-4ca1a06b3665 None admin] PATH_INFO: `/v3/services` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:39.702446 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6610de3-3452-4fb1-90b8-4ca1a06b3665 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:39.703786 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6610de3-3452-4fb1-90b8-4ca1a06b3665 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:39.710455 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 649/1301] 10.222.0.16 () {56 vars in 1087 bytes} [Wed Aug 31 10:48:39 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:48:41.619285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683cc8d-1af8-424e-aad2-a5253b6a906c None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:41.619845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683cc8d-1af8-424e-aad2-a5253b6a906c None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:41.619845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683cc8d-1af8-424e-aad2-a5253b6a906c None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:41.621135 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 653/1302] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:48:41.626696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af8b92-dd76-4c88-af24-7994aff6a8f8 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:41.627111 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af8b92-dd76-4c88-af24-7994aff6a8f8 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:41.627242 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af8b92-dd76-4c88-af24-7994aff6a8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:41.646832 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-01af8b92-dd76-4c88-af24-7994aff6a8f8 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:48:41.649148 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-01af8b92-dd76-4c88-af24-7994aff6a8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:48:41.714016 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 650/1303] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:48:41 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:48:41.982359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683cc8d-1af8-424e-aad2-a5253b6a906c None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:42.076908 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:42.083208 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683cc8d-1af8-424e-aad2-a5253b6a906c None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:42.120332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbba3b6-ec15-4135-a533-3921b772f525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:42.120957 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbba3b6-ec15-4135-a533-3921b772f525 None admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:42.121313 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbba3b6-ec15-4135-a533-3921b772f525 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:42.121613 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbba3b6-ec15-4135-a533-3921b772f525 None admin] PATH_INFO: `/v3/services/placement` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:42.122267 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbba3b6-ec15-4135-a533-3921b772f525 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:42.123669 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbba3b6-ec15-4135-a533-3921b772f525 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:42.132741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application [None req-7fbba3b6-ec15-4135-a533-3921b772f525 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 10:48:42.132741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:42.132741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:48:42.132741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:48:42.132741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:48:42.132741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:48:42.132741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:48:42.132741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:48:42.132741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 10:48:42.132741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:48:42.132741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:48:42.132741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:48:42.132741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:48:42.132741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:48:42.132741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:48:42.132741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:48:42.132741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:48:42.133873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:48:42.133873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:48:42.133873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:48:42.133873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:48:42.133873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:48:42.133873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:48:42.133873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:48:42.133873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:48:42.133873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:48:42.133873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:48:42.133873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:48:42.133873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:48:42.133873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:48:42.133873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:48:42.133873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:48:42.133873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:48:42.134818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:48:42.134818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application with Lock( Aug 31 10:48:42.134818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:48:42.134818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:48:42.134818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:48:42.134818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:48:42.134818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:48:42.134818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:48:42.134818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:48:42.134818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:48:42.134818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:48:42.134818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:48:42.134818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 10:48:42.134818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: ERROR keystone.server.flask.application Aug 31 10:48:42.134818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 654/1304] 10.222.0.16 () {56 vars in 1117 bytes} [Wed Aug 31 10:48:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:48:42.141427 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af8b92-dd76-4c88-af24-7994aff6a8f8 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:42.216235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:42.239664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af8b92-dd76-4c88-af24-7994aff6a8f8 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:42.273283 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb3818-63c9-4dec-b1d2-e5885fd6b2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:42.276581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb3818-63c9-4dec-b1d2-e5885fd6b2e5 None admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:42.276581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb3818-63c9-4dec-b1d2-e5885fd6b2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:42.276581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb3818-63c9-4dec-b1d2-e5885fd6b2e5 None admin] PATH_INFO: `/v3/services` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:42.276581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb3818-63c9-4dec-b1d2-e5885fd6b2e5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:48:42.276581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb3818-63c9-4dec-b1d2-e5885fd6b2e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:42.277034 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb3818-63c9-4dec-b1d2-e5885fd6b2e5 None admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:42.282712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 651/1305] 10.222.0.16 () {56 vars in 1116 bytes} [Wed Aug 31 10:48:42 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:48:42.288628 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbba3b6-ec15-4135-a533-3921b772f525 None admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:48:42.390348 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbba3b6-ec15-4135-a533-3921b772f525 None admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:48:42.423338 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c0929c-8f88-4205-be38-f9f9dcaacff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:48:42.423926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0929c-8f88-4205-be38-f9f9dcaacff1 None admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:48:42.424200 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0929c-8f88-4205-be38-f9f9dcaacff1 None admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:48:42.424466 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0929c-8f88-4205-be38-f9f9dcaacff1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:48:42.425129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c0929c-8f88-4205-be38-f9f9dcaacff1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:48:42.426557 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c0929c-8f88-4205-be38-f9f9dcaacff1 None admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:48:42.516432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 655/1306] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 10:48:42 2022] POST /identity/v3/endpoints => generated 331 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:48:52.279689 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=30 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:48:52.510144 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:54:56.779194 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16105a33-72f3-4460-bd51-9005a6db8a58 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:56.780436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16105a33-72f3-4460-bd51-9005a6db8a58 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:56.780436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16105a33-72f3-4460-bd51-9005a6db8a58 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:56.781892 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 652/1307] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:54:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:54:56.787997 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29345ea-af9a-4a73-97b1-4bb27978c827 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:56.788433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29345ea-af9a-4a73-97b1-4bb27978c827 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:56.788520 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29345ea-af9a-4a73-97b1-4bb27978c827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:56.924700 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-e29345ea-af9a-4a73-97b1-4bb27978c827 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:54:56.926555 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-e29345ea-af9a-4a73-97b1-4bb27978c827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:54:57.070430 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 656/1308] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:54:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:54:57.380667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d670cc-4591-48e0-8f49-141e541e8a4a None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:57.381125 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d670cc-4591-48e0-8f49-141e541e8a4a None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:57.381765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d670cc-4591-48e0-8f49-141e541e8a4a None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:57.383695 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 653/1309] 10.222.0.16 () {54 vars in 965 bytes} [Wed Aug 31 10:54:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:54:57.390743 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc25446-7b65-47e3-8b64-9933c94ac982 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:57.391453 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc25446-7b65-47e3-8b64-9933c94ac982 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:57.392096 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc25446-7b65-47e3-8b64-9933c94ac982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:57.502008 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-ccc25446-7b65-47e3-8b64-9933c94ac982 None None] MFA Rules not processed for user `1a00b7f45b2e47b8a01048c36d00f742`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:54:57.503270 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-ccc25446-7b65-47e3-8b64-9933c94ac982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:54:57.526328 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:54:57.593495 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 657/1310] 10.222.0.16 () {58 vars in 1062 bytes} [Wed Aug 31 10:54:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:54:57.602835 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d670cc-4591-48e0-8f49-141e541e8a4a None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:54:57.605161 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-b1d670cc-4591-48e0-8f49-141e541e8a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:54:57.731036 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d670cc-4591-48e0-8f49-141e541e8a4a None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:54:57.812445 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6466879-e076-470c-aa50-38f78864c47f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:54:57.813862 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6466879-e076-470c-aa50-38f78864c47f service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:57.814294 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6466879-e076-470c-aa50-38f78864c47f service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:57.814738 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6466879-e076-470c-aa50-38f78864c47f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:57.816531 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-b6466879-e076-470c-aa50-38f78864c47f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:54:57.878764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:54:57.940040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6466879-e076-470c-aa50-38f78864c47f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:54:57.941509 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6466879-e076-470c-aa50-38f78864c47f service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:54:57.966438 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 654/1311] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:54:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:55:00.342365 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8c73f-5291-4451-a9fd-13a398def98e None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:00.342915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8c73f-5291-4451-a9fd-13a398def98e None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:00.342915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8c73f-5291-4451-a9fd-13a398def98e None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:00.344316 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 658/1312] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:55:00.349698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae8702-ec93-40b5-82bd-ceec60c63f3f None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:00.350108 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae8702-ec93-40b5-82bd-ceec60c63f3f None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:00.350337 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae8702-ec93-40b5-82bd-ceec60c63f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:00.432232 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-53ae8702-ec93-40b5-82bd-ceec60c63f3f None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:00.433458 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-53ae8702-ec93-40b5-82bd-ceec60c63f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:00.534461 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 655/1313] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:55:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:55:00.817284 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8c73f-5291-4451-a9fd-13a398def98e None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:00.818871 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-8df8c73f-5291-4451-a9fd-13a398def98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:00.921837 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8c73f-5291-4451-a9fd-13a398def98e None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:00.990313 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:00.992654 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720859b-9f3c-4072-aa3e-0de243979380 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:00.993250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720859b-9f3c-4072-aa3e-0de243979380 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:00.993523 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720859b-9f3c-4072-aa3e-0de243979380 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:00.993802 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720859b-9f3c-4072-aa3e-0de243979380 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:00.995403 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-7720859b-9f3c-4072-aa3e-0de243979380 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:01.103235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7720859b-9f3c-4072-aa3e-0de243979380 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:01.104753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7720859b-9f3c-4072-aa3e-0de243979380 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:01.124078 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 659/1314] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:55:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:55:03.598148 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6443d-d05b-4e87-a220-9ba05e2db765 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:03.598705 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6443d-d05b-4e87-a220-9ba05e2db765 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:03.598825 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6443d-d05b-4e87-a220-9ba05e2db765 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:03.600072 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 656/1315] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:55:03.605355 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168395f9-9fd3-4ea2-a994-031b2fc3b00c None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:03.605816 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168395f9-9fd3-4ea2-a994-031b2fc3b00c None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:03.605942 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168395f9-9fd3-4ea2-a994-031b2fc3b00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:03.671248 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-168395f9-9fd3-4ea2-a994-031b2fc3b00c None None] MFA Rules not processed for user `e3f891d792a74027848d05c2a300e2af`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:03.672444 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-168395f9-9fd3-4ea2-a994-031b2fc3b00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:03.723624 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:03.767934 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 660/1316] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:55:03 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:55:04.081708 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ae920-947f-4c1d-9010-30feb2e5eed7 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:04.081994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ae920-947f-4c1d-9010-30feb2e5eed7 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:04.082274 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ae920-947f-4c1d-9010-30feb2e5eed7 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:04.083661 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 657/1317] 10.222.0.16 () {54 vars in 965 bytes} [Wed Aug 31 10:55:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:55:04.089959 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681a598-d574-4b59-ad84-51cc9d10aae8 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:04.090250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681a598-d574-4b59-ad84-51cc9d10aae8 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:04.090554 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681a598-d574-4b59-ad84-51cc9d10aae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:04.169367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-5681a598-d574-4b59-ad84-51cc9d10aae8 None None] MFA Rules not processed for user `1a00b7f45b2e47b8a01048c36d00f742`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:04.171112 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-5681a598-d574-4b59-ad84-51cc9d10aae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:04.247486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 661/1318] 10.222.0.16 () {58 vars in 1062 bytes} [Wed Aug 31 10:55:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:55:04.257832 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ae920-947f-4c1d-9010-30feb2e5eed7 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:04.259488 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-1e2ae920-947f-4c1d-9010-30feb2e5eed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:04.269770 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:04.366526 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ae920-947f-4c1d-9010-30feb2e5eed7 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:04.438981 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe62d17-1326-4c31-905a-a7e0edc04076 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:04.439578 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe62d17-1326-4c31-905a-a7e0edc04076 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:04.439850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe62d17-1326-4c31-905a-a7e0edc04076 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:04.440116 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe62d17-1326-4c31-905a-a7e0edc04076 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:04.441712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-8fe62d17-1326-4c31-905a-a7e0edc04076 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:04.503795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:04.550096 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe62d17-1326-4c31-905a-a7e0edc04076 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:04.551553 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe62d17-1326-4c31-905a-a7e0edc04076 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:04.572176 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 658/1319] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:55:04 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:55:07.606418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cfbbd0-eba9-4184-9ab8-f958c98f6ec1 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:07.607030 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cfbbd0-eba9-4184-9ab8-f958c98f6ec1 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:07.607030 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cfbbd0-eba9-4184-9ab8-f958c98f6ec1 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:07.609214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 662/1320] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:55:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:55:07.614380 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fb8b0-22bb-44ab-8acd-00d8ad802df3 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:07.614848 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fb8b0-22bb-44ab-8acd-00d8ad802df3 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:07.614979 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fb8b0-22bb-44ab-8acd-00d8ad802df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:07.678795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-423fb8b0-22bb-44ab-8acd-00d8ad802df3 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:07.680013 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-423fb8b0-22bb-44ab-8acd-00d8ad802df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:07.773099 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 659/1321] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:55:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:55:08.054225 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cfbbd0-eba9-4184-9ab8-f958c98f6ec1 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:08.055795 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-19cfbbd0-eba9-4184-9ab8-f958c98f6ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:08.107331 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:08.163736 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cfbbd0-eba9-4184-9ab8-f958c98f6ec1 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:08.240211 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab8acf-e018-408b-8cad-88e01aa0997b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:08.240812 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab8acf-e018-408b-8cad-88e01aa0997b service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:08.241089 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab8acf-e018-408b-8cad-88e01aa0997b service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:08.241360 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab8acf-e018-408b-8cad-88e01aa0997b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:08.242985 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-30ab8acf-e018-408b-8cad-88e01aa0997b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:08.343340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:08.357292 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ab8acf-e018-408b-8cad-88e01aa0997b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:08.359356 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ab8acf-e018-408b-8cad-88e01aa0997b service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:08.379344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 663/1322] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:55:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:55:11.348963 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d1492-9d0e-4d09-ba67-f12681184bcf None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:11.349497 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d1492-9d0e-4d09-ba67-f12681184bcf None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:11.349497 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d1492-9d0e-4d09-ba67-f12681184bcf None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:11.350995 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 660/1323] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:55:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:55:11.356547 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeff014-d8e5-44ed-a60a-94cccbe98b34 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:11.357055 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeff014-d8e5-44ed-a60a-94cccbe98b34 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:11.357502 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeff014-d8e5-44ed-a60a-94cccbe98b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:11.429444 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-faeff014-d8e5-44ed-a60a-94cccbe98b34 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:11.430803 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-faeff014-d8e5-44ed-a60a-94cccbe98b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:11.538492 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 664/1324] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:55:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:55:11.827194 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d1492-9d0e-4d09-ba67-f12681184bcf None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:11.828735 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-1a0d1492-9d0e-4d09-ba67-f12681184bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:11.849514 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:11.940778 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d1492-9d0e-4d09-ba67-f12681184bcf None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:12.015625 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e6f8d-c816-4979-8780-8bdb2a085fb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:12.016229 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e6f8d-c816-4979-8780-8bdb2a085fb2 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:12.016503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e6f8d-c816-4979-8780-8bdb2a085fb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:12.016769 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e6f8d-c816-4979-8780-8bdb2a085fb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:12.018354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a47e6f8d-c816-4979-8780-8bdb2a085fb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:12.090466 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:12.131266 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47e6f8d-c816-4979-8780-8bdb2a085fb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:12.132700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47e6f8d-c816-4979-8780-8bdb2a085fb2 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:12.151368 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 661/1325] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:55:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:55:14.230935 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696245b9-6a42-40fd-a93a-3f3fbb7097ec None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:14.231536 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696245b9-6a42-40fd-a93a-3f3fbb7097ec None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:14.231536 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696245b9-6a42-40fd-a93a-3f3fbb7097ec None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:14.233816 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 665/1326] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:55:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:55:14.239828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba52aa2-02f9-48a7-889c-a9efa9e173ce None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:14.240316 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba52aa2-02f9-48a7-889c-a9efa9e173ce None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:14.240485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba52aa2-02f9-48a7-889c-a9efa9e173ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:14.309415 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-5ba52aa2-02f9-48a7-889c-a9efa9e173ce None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:14.311187 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-5ba52aa2-02f9-48a7-889c-a9efa9e173ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:14.412455 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 662/1327] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:55:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:55:14.701615 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696245b9-6a42-40fd-a93a-3f3fbb7097ec None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:14.703209 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-696245b9-6a42-40fd-a93a-3f3fbb7097ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:14.771908 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:14.816233 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696245b9-6a42-40fd-a93a-3f3fbb7097ec None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:14.893865 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f0c65-abd7-43a0-946f-ea110dfc9d63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:14.894592 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f0c65-abd7-43a0-946f-ea110dfc9d63 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:14.894914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f0c65-abd7-43a0-946f-ea110dfc9d63 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:14.895226 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f0c65-abd7-43a0-946f-ea110dfc9d63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:14.897047 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-930f0c65-abd7-43a0-946f-ea110dfc9d63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:15.011882 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:15.015733 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f0c65-abd7-43a0-946f-ea110dfc9d63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:15.017395 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f0c65-abd7-43a0-946f-ea110dfc9d63 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:15.036939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 666/1328] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:55:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:55:17.027505 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc5359-3733-4789-a85a-9e1161063770 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:17.028057 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc5359-3733-4789-a85a-9e1161063770 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:17.028057 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc5359-3733-4789-a85a-9e1161063770 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:17.030046 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 663/1329] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:55:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:55:17.035358 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d32b8-dc0f-4025-9c38-0292559a7e1f None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:17.035794 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d32b8-dc0f-4025-9c38-0292559a7e1f None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:17.035897 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d32b8-dc0f-4025-9c38-0292559a7e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:17.101559 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-1b3d32b8-dc0f-4025-9c38-0292559a7e1f None None] MFA Rules not processed for user `e3f891d792a74027848d05c2a300e2af`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:17.102795 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-1b3d32b8-dc0f-4025-9c38-0292559a7e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:17.201707 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 667/1330] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:55:17 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:55:17.483381 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc5359-3733-4789-a85a-9e1161063770 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:17.485205 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-adfc5359-3733-4789-a85a-9e1161063770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:17.517167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:17.590273 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc5359-3733-4789-a85a-9e1161063770 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:17.665649 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896e872-d397-4b40-b0fe-66c92f81cf60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:17.666181 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896e872-d397-4b40-b0fe-66c92f81cf60 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:17.666493 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896e872-d397-4b40-b0fe-66c92f81cf60 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:17.666763 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896e872-d397-4b40-b0fe-66c92f81cf60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:17.668384 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-d896e872-d397-4b40-b0fe-66c92f81cf60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:17.744831 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:17.777754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d896e872-d397-4b40-b0fe-66c92f81cf60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:17.779858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d896e872-d397-4b40-b0fe-66c92f81cf60 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:17.799294 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 664/1331] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:55:17 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:55:20.210171 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b7aeb-147d-4a21-92ba-6398de8c135f None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:20.210745 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b7aeb-147d-4a21-92ba-6398de8c135f None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:20.210745 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b7aeb-147d-4a21-92ba-6398de8c135f None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:20.212098 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 668/1332] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:55:20.217815 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc6076-d0a6-49bf-9a46-9f8391f97b57 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:20.218328 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc6076-d0a6-49bf-9a46-9f8391f97b57 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:20.218328 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc6076-d0a6-49bf-9a46-9f8391f97b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:20.284044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-96fc6076-d0a6-49bf-9a46-9f8391f97b57 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:20.285248 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-96fc6076-d0a6-49bf-9a46-9f8391f97b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:20.384165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 665/1333] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:55:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:55:20.666138 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b7aeb-147d-4a21-92ba-6398de8c135f None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:20.667765 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-f35b7aeb-147d-4a21-92ba-6398de8c135f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:20.738291 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:20.772872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b7aeb-147d-4a21-92ba-6398de8c135f None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:20.851440 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130506c-dd17-4b76-b360-4c652cc81cc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:20.852045 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130506c-dd17-4b76-b360-4c652cc81cc7 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:20.852320 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130506c-dd17-4b76-b360-4c652cc81cc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:20.852584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130506c-dd17-4b76-b360-4c652cc81cc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:20.854179 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-8130506c-dd17-4b76-b360-4c652cc81cc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:20.962843 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130506c-dd17-4b76-b360-4c652cc81cc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:20.964295 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130506c-dd17-4b76-b360-4c652cc81cc7 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:20.974207 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:20.983720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 669/1334] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:55:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:55:22.954075 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8e9a4-3730-4489-a741-3837823eb39d None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:22.954641 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8e9a4-3730-4489-a741-3837823eb39d None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:22.954641 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8e9a4-3730-4489-a741-3837823eb39d None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:22.956001 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 666/1335] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:55:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:55:22.961078 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ded8b2-2174-49db-8207-6fbac6800d99 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:22.961475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ded8b2-2174-49db-8207-6fbac6800d99 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:22.961628 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ded8b2-2174-49db-8207-6fbac6800d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:23.026550 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-16ded8b2-2174-49db-8207-6fbac6800d99 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:23.027752 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-16ded8b2-2174-49db-8207-6fbac6800d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:23.125222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 670/1336] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:55:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:55:23.420124 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8e9a4-3730-4489-a741-3837823eb39d None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:23.421669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-13b8e9a4-3730-4489-a741-3837823eb39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:23.462038 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:23.527219 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8e9a4-3730-4489-a741-3837823eb39d None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:23.603594 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9cd31f-d999-4656-b3a6-421ed2876e83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:23.604199 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cd31f-d999-4656-b3a6-421ed2876e83 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:23.604475 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cd31f-d999-4656-b3a6-421ed2876e83 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:23.604742 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cd31f-d999-4656-b3a6-421ed2876e83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:23.606358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-0b9cd31f-d999-4656-b3a6-421ed2876e83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:23.692630 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:23.717669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9cd31f-d999-4656-b3a6-421ed2876e83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:23.719194 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9cd31f-d999-4656-b3a6-421ed2876e83 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:23.738590 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 667/1337] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:55:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:55:26.374935 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6e122-7dde-4f08-8eb1-bc25a98f3934 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:26.375440 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6e122-7dde-4f08-8eb1-bc25a98f3934 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:26.375440 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6e122-7dde-4f08-8eb1-bc25a98f3934 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:26.376720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 671/1338] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:55:26.381964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0033b-43cd-4482-b3a7-7b5c5bf32ed2 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:26.382416 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0033b-43cd-4482-b3a7-7b5c5bf32ed2 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:26.382544 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0033b-43cd-4482-b3a7-7b5c5bf32ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:26.448298 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-dff0033b-43cd-4482-b3a7-7b5c5bf32ed2 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:26.449492 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-dff0033b-43cd-4482-b3a7-7b5c5bf32ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:26.544930 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 668/1339] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:55:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:55:26.826803 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6e122-7dde-4f08-8eb1-bc25a98f3934 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:26.828346 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-7cb6e122-7dde-4f08-8eb1-bc25a98f3934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:26.905720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:26.933423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6e122-7dde-4f08-8eb1-bc25a98f3934 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:27.008395 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4102536-d9ae-4d6d-8f61-5dc11a46dff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:27.008995 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4102536-d9ae-4d6d-8f61-5dc11a46dff8 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:27.009270 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4102536-d9ae-4d6d-8f61-5dc11a46dff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:27.009534 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4102536-d9ae-4d6d-8f61-5dc11a46dff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:27.011141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-b4102536-d9ae-4d6d-8f61-5dc11a46dff8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:27.125142 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4102536-d9ae-4d6d-8f61-5dc11a46dff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:27.126605 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4102536-d9ae-4d6d-8f61-5dc11a46dff8 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:27.145118 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 672/1340] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:55:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:55:31.317329 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477eb54-b58f-43c5-a8ea-015245d314f3 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:31.317889 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477eb54-b58f-43c5-a8ea-015245d314f3 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:31.317889 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477eb54-b58f-43c5-a8ea-015245d314f3 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:31.319393 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 669/1341] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:55:31.324900 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f87c39-78ce-493a-afea-335be3544cb4 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:31.330122 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f87c39-78ce-493a-afea-335be3544cb4 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:31.330122 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f87c39-78ce-493a-afea-335be3544cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:31.331168 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:31.394309 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-43f87c39-78ce-493a-afea-335be3544cb4 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:31.395470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-43f87c39-78ce-493a-afea-335be3544cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:31.498776 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 673/1342] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:55:31.782457 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477eb54-b58f-43c5-a8ea-015245d314f3 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:31.784014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-1477eb54-b58f-43c5-a8ea-015245d314f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:31.836185 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:31.893996 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477eb54-b58f-43c5-a8ea-015245d314f3 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:31.971914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc3891-d0e8-4011-afe4-bb2ebb4378b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:31.972525 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc3891-d0e8-4011-afe4-bb2ebb4378b3 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:31.972804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc3891-d0e8-4011-afe4-bb2ebb4378b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:31.973073 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc3891-d0e8-4011-afe4-bb2ebb4378b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:31.974700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-98cc3891-d0e8-4011-afe4-bb2ebb4378b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:32.072116 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:32.088404 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc3891-d0e8-4011-afe4-bb2ebb4378b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:32.089838 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc3891-d0e8-4011-afe4-bb2ebb4378b3 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:32.111393 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 670/1343] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:55:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 10:55:35.232542 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c75201-daa5-4bfc-9984-09355ce38331 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:35.233073 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c75201-daa5-4bfc-9984-09355ce38331 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:35.233073 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c75201-daa5-4bfc-9984-09355ce38331 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:35.234842 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 674/1344] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:55:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:55:35.239895 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b18aa3-394f-4db4-ab24-8efc9c13e00a None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:35.240295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b18aa3-394f-4db4-ab24-8efc9c13e00a None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:35.241165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b18aa3-394f-4db4-ab24-8efc9c13e00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:35.308549 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-a2b18aa3-394f-4db4-ab24-8efc9c13e00a None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:35.310284 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a2b18aa3-394f-4db4-ab24-8efc9c13e00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:35.410961 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 671/1345] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:55:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:55:35.706923 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c75201-daa5-4bfc-9984-09355ce38331 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:35.708490 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-e2c75201-daa5-4bfc-9984-09355ce38331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:35.803824 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:35.821552 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c75201-daa5-4bfc-9984-09355ce38331 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:35.912337 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d2cc5d-87bf-41b2-bc6f-685d9c5b4ce0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:35.912956 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2cc5d-87bf-41b2-bc6f-685d9c5b4ce0 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:35.913232 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2cc5d-87bf-41b2-bc6f-685d9c5b4ce0 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:35.913502 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2cc5d-87bf-41b2-bc6f-685d9c5b4ce0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:35.915223 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-52d2cc5d-87bf-41b2-bc6f-685d9c5b4ce0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:36.027078 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d2cc5d-87bf-41b2-bc6f-685d9c5b4ce0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:36.029238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d2cc5d-87bf-41b2-bc6f-685d9c5b4ce0 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:36.049049 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 675/1346] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:55:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:55:40.525314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d328cd1-8bc4-4b0f-9651-e2a09e353885 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:40.525834 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d328cd1-8bc4-4b0f-9651-e2a09e353885 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:40.525834 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d328cd1-8bc4-4b0f-9651-e2a09e353885 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:40.527178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 672/1347] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:55:40.532936 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6a643-26ca-4d32-a62f-45ce014af82c None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:40.533426 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6a643-26ca-4d32-a62f-45ce014af82c None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:40.533523 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6a643-26ca-4d32-a62f-45ce014af82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:40.549760 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:40.605128 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-71d6a643-26ca-4d32-a62f-45ce014af82c None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:40.606468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-71d6a643-26ca-4d32-a62f-45ce014af82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:40.721890 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 676/1348] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:55:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:55:41.006295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d328cd1-8bc4-4b0f-9651-e2a09e353885 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:41.007870 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-8d328cd1-8bc4-4b0f-9651-e2a09e353885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:41.070055 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:41.116923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d328cd1-8bc4-4b0f-9651-e2a09e353885 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:41.189941 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0037bc-ac98-4653-8c54-69897af5a772 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:41.190578 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0037bc-ac98-4653-8c54-69897af5a772 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:41.190905 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0037bc-ac98-4653-8c54-69897af5a772 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:41.191180 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0037bc-ac98-4653-8c54-69897af5a772 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:41.192801 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-3b0037bc-ac98-4653-8c54-69897af5a772 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:41.302950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:41.306367 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0037bc-ac98-4653-8c54-69897af5a772 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:41.307859 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0037bc-ac98-4653-8c54-69897af5a772 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:41.327571 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 673/1349] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:55:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:55:46.324478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bad17-de29-42c8-9baf-fac658ce49b9 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:46.325545 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bad17-de29-42c8-9baf-fac658ce49b9 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:46.325788 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bad17-de29-42c8-9baf-fac658ce49b9 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:46.327469 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 677/1350] 10.222.0.16 () {54 vars in 921 bytes} [Wed Aug 31 10:55:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:55:46.332993 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c17329-57bd-49ce-bd06-a1ec85456d49 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:46.333542 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c17329-57bd-49ce-bd06-a1ec85456d49 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:46.333542 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c17329-57bd-49ce-bd06-a1ec85456d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:46.442858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-b0c17329-57bd-49ce-bd06-a1ec85456d49 None None] MFA Rules not processed for user `f3db46a4b4e74cc0bb2a7bd6f0e99105`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:46.444360 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-b0c17329-57bd-49ce-bd06-a1ec85456d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:46.625142 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 674/1351] 10.222.0.16 () {58 vars in 1018 bytes} [Wed Aug 31 10:55:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:55:50.719522 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=59 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:51.068344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3204e-0f8c-40ca-935b-04824dce0d9a None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:51.068611 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3204e-0f8c-40ca-935b-04824dce0d9a None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:51.068886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3204e-0f8c-40ca-935b-04824dce0d9a None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:51.070507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 678/1352] 10.222.0.16 () {54 vars in 921 bytes} [Wed Aug 31 10:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:55:51.077255 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5089988-d7d3-4d33-b490-5c5aa336978a None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:51.078377 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5089988-d7d3-4d33-b490-5c5aa336978a None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:51.078834 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5089988-d7d3-4d33-b490-5c5aa336978a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:51.150195 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:51.174147 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-a5089988-d7d3-4d33-b490-5c5aa336978a None None] MFA Rules not processed for user `f3db46a4b4e74cc0bb2a7bd6f0e99105`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:51.178703 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a5089988-d7d3-4d33-b490-5c5aa336978a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:51.305424 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 675/1353] 10.222.0.16 () {58 vars in 1018 bytes} [Wed Aug 31 10:55:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:55:52.426624 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e2942-d292-4f1b-bf07-f5e4f9e6fe52 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:52.427238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e2942-d292-4f1b-bf07-f5e4f9e6fe52 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:52.427238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e2942-d292-4f1b-bf07-f5e4f9e6fe52 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:52.428714 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 679/1354] 10.222.0.16 () {54 vars in 921 bytes} [Wed Aug 31 10:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:55:52.433728 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7a594-5ce5-4d75-b152-71a34b7753cd None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:52.434782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7a594-5ce5-4d75-b152-71a34b7753cd None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:52.434898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7a594-5ce5-4d75-b152-71a34b7753cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:52.511775 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-2bc7a594-5ce5-4d75-b152-71a34b7753cd None None] MFA Rules not processed for user `f3db46a4b4e74cc0bb2a7bd6f0e99105`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:52.513603 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-2bc7a594-5ce5-4d75-b152-71a34b7753cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:52.555439 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:52.620531 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 676/1355] 10.222.0.16 () {58 vars in 1018 bytes} [Wed Aug 31 10:55:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:55:56.154256 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55878d98-6fea-4ee7-acee-d7d4983c627d None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:56.154813 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55878d98-6fea-4ee7-acee-d7d4983c627d None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:56.154813 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55878d98-6fea-4ee7-acee-d7d4983c627d None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:56.156159 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 680/1356] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:55:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:55:56.161313 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cef775-43dc-471a-aa91-cb942c76ee65 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:56.161755 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cef775-43dc-471a-aa91-cb942c76ee65 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:56.161852 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cef775-43dc-471a-aa91-cb942c76ee65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:56.250018 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-99cef775-43dc-471a-aa91-cb942c76ee65 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:56.251340 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-99cef775-43dc-471a-aa91-cb942c76ee65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:56.404342 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 677/1357] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:55:56.714658 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61888e75-b816-4056-8d48-47d1754cad57 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:56.715601 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61888e75-b816-4056-8d48-47d1754cad57 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:56.715875 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61888e75-b816-4056-8d48-47d1754cad57 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:56.717228 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 681/1358] 10.222.0.16 () {54 vars in 962 bytes} [Wed Aug 31 10:55:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:55:56.726325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe147fa-e487-41b7-9697-444e91150815 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:56.726852 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe147fa-e487-41b7-9697-444e91150815 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:56.727343 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe147fa-e487-41b7-9697-444e91150815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:56.748756 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:56.814059 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-cfe147fa-e487-41b7-9697-444e91150815 None None] MFA Rules not processed for user `f11971a594014d86968c3356d11d42f7`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:56.815443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-cfe147fa-e487-41b7-9697-444e91150815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:56.952266 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 678/1359] 10.222.0.16 () {58 vars in 1059 bytes} [Wed Aug 31 10:55:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:55:56.961486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61888e75-b816-4056-8d48-47d1754cad57 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:56.963402 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-61888e75-b816-4056-8d48-47d1754cad57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:57.138771 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61888e75-b816-4056-8d48-47d1754cad57 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:57.216536 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:57.262324 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bad989-eafa-4c43-8dff-4f055d32a59c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:57.263508 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bad989-eafa-4c43-8dff-4f055d32a59c service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:57.264051 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bad989-eafa-4c43-8dff-4f055d32a59c service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:57.264614 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bad989-eafa-4c43-8dff-4f055d32a59c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:57.266476 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-58bad989-eafa-4c43-8dff-4f055d32a59c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:57.364769 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bad989-eafa-4c43-8dff-4f055d32a59c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:57.366332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bad989-eafa-4c43-8dff-4f055d32a59c service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:57.386373 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 682/1360] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:55:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:55:59.571066 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d39e7-bc7d-4c73-8788-465cb009fdc7 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:59.571579 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d39e7-bc7d-4c73-8788-465cb009fdc7 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:59.571579 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d39e7-bc7d-4c73-8788-465cb009fdc7 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:59.573034 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 679/1361] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:55:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:55:59.578516 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99adb5-fccf-4358-9d1d-a9b7aa7806c7 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:59.578968 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99adb5-fccf-4358-9d1d-a9b7aa7806c7 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:59.579066 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99adb5-fccf-4358-9d1d-a9b7aa7806c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:59.647276 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-8d99adb5-fccf-4358-9d1d-a9b7aa7806c7 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:59.648468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-8d99adb5-fccf-4358-9d1d-a9b7aa7806c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:59.691294 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:59.766138 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 683/1362] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:56:00.069073 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d39e7-bc7d-4c73-8788-465cb009fdc7 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:56:00.070901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-403d39e7-bc7d-4c73-8788-465cb009fdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:00.145425 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:00.232187 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d39e7-bc7d-4c73-8788-465cb009fdc7 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:56:00.372312 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089260de-c74c-4382-8ab6-e38769d82f78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:56:00.372312 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089260de-c74c-4382-8ab6-e38769d82f78 service nova] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:00.372312 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089260de-c74c-4382-8ab6-e38769d82f78 service nova] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:00.372312 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089260de-c74c-4382-8ab6-e38769d82f78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:00.372861 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-089260de-c74c-4382-8ab6-e38769d82f78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:00.400070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:00.475306 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089260de-c74c-4382-8ab6-e38769d82f78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:56:00.477276 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089260de-c74c-4382-8ab6-e38769d82f78 service nova] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:56:00.500739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 680/1363] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:56:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:56:02.942873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2918a-70c8-4ed2-9548-4f12470ea6c4 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:02.943411 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2918a-70c8-4ed2-9548-4f12470ea6c4 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:02.943411 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2918a-70c8-4ed2-9548-4f12470ea6c4 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:02.944806 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 684/1364] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:56:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:56:02.951073 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd04a0-f266-440c-9059-af1d864782f6 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:02.951536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd04a0-f266-440c-9059-af1d864782f6 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:02.951639 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd04a0-f266-440c-9059-af1d864782f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:03.024999 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-d5bd04a0-f266-440c-9059-af1d864782f6 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:56:03.026218 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-d5bd04a0-f266-440c-9059-af1d864782f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:03.132529 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:03.137973 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 681/1365] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:56:03.435471 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fac45-452c-4cf1-b82b-d217165df608 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:03.435981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fac45-452c-4cf1-b82b-d217165df608 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:03.436384 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fac45-452c-4cf1-b82b-d217165df608 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:03.437804 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 685/1366] 10.222.0.16 () {54 vars in 962 bytes} [Wed Aug 31 10:56:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:56:03.443446 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a87421-74ad-4e0c-9858-91f5c9a3af9b None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:03.443832 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a87421-74ad-4e0c-9858-91f5c9a3af9b None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:03.444225 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a87421-74ad-4e0c-9858-91f5c9a3af9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:03.532459 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-a1a87421-74ad-4e0c-9858-91f5c9a3af9b None None] MFA Rules not processed for user `f11971a594014d86968c3356d11d42f7`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:56:03.533836 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a1a87421-74ad-4e0c-9858-91f5c9a3af9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:03.675371 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 682/1367] 10.222.0.16 () {58 vars in 1059 bytes} [Wed Aug 31 10:56:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:56:03.683913 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18fac45-452c-4cf1-b82b-d217165df608 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:56:03.689716 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-e18fac45-452c-4cf1-b82b-d217165df608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:03.845755 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18fac45-452c-4cf1-b82b-d217165df608 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:56:03.872173 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:03.975784 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4652e6-e9a3-4e5d-8385-63e7b1bd0eba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:56:03.976441 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4652e6-e9a3-4e5d-8385-63e7b1bd0eba service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:03.976698 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4652e6-e9a3-4e5d-8385-63e7b1bd0eba service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:03.976814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4652e6-e9a3-4e5d-8385-63e7b1bd0eba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:03.978438 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-cc4652e6-e9a3-4e5d-8385-63e7b1bd0eba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:04.078637 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4652e6-e9a3-4e5d-8385-63e7b1bd0eba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:56:04.080242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4652e6-e9a3-4e5d-8385-63e7b1bd0eba service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:56:04.110783 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 686/1368] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:56:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:56:04.450742 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0dbae-a4a4-49d6-abbc-a273ed1ca6ae None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:04.451451 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0dbae-a4a4-49d6-abbc-a273ed1ca6ae None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:04.451451 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0dbae-a4a4-49d6-abbc-a273ed1ca6ae None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:04.452690 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 683/1369] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:56:04.459029 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa83b5-92e7-4fca-a465-4489d0178d2f None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:04.459423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa83b5-92e7-4fca-a465-4489d0178d2f None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:04.459706 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa83b5-92e7-4fca-a465-4489d0178d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:04.503798 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:04.556423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-47aa83b5-92e7-4fca-a465-4489d0178d2f None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:56:04.557925 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-47aa83b5-92e7-4fca-a465-4489d0178d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:04.682502 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 687/1370] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:56:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:56:05.269117 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea3e69-d941-43ab-81f3-7ca4a01b3058 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:05.269117 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea3e69-d941-43ab-81f3-7ca4a01b3058 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:05.269117 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea3e69-d941-43ab-81f3-7ca4a01b3058 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:05.271126 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 684/1371] 10.222.0.16 () {54 vars in 964 bytes} [Wed Aug 31 10:56:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:56:05.275429 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e1cf6-3c6b-4924-aec5-d36b2ea25559 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:05.275992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e1cf6-3c6b-4924-aec5-d36b2ea25559 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:05.275992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e1cf6-3c6b-4924-aec5-d36b2ea25559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:05.360915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-416e1cf6-3c6b-4924-aec5-d36b2ea25559 None None] MFA Rules not processed for user `49f9aeb9857b45d68c3baa8fc4acc5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:56:05.362244 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-416e1cf6-3c6b-4924-aec5-d36b2ea25559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:05.414477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:05.445461 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 688/1372] 10.222.0.16 () {58 vars in 1061 bytes} [Wed Aug 31 10:56:05 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:56:05.453830 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea3e69-d941-43ab-81f3-7ca4a01b3058 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:56:05.455379 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a2ea3e69-d941-43ab-81f3-7ca4a01b3058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:05.512767 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:05.576106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea3e69-d941-43ab-81f3-7ca4a01b3058 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:56:05.663609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92aa54d-f0fd-4f3c-bbfc-e9165e3bd011 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:56:05.664266 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92aa54d-f0fd-4f3c-bbfc-e9165e3bd011 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:05.664540 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92aa54d-f0fd-4f3c-bbfc-e9165e3bd011 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:05.664801 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92aa54d-f0fd-4f3c-bbfc-e9165e3bd011 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:05.666406 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-f92aa54d-f0fd-4f3c-bbfc-e9165e3bd011 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:05.770639 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:05.788090 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92aa54d-f0fd-4f3c-bbfc-e9165e3bd011 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:56:05.789160 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92aa54d-f0fd-4f3c-bbfc-e9165e3bd011 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:56:05.812528 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 685/1373] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:56:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:56:06.347843 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e3318-857c-4689-877f-254c07f0a963 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:06.348405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e3318-857c-4689-877f-254c07f0a963 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:06.348556 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e3318-857c-4689-877f-254c07f0a963 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:06.349968 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 689/1374] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:56:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:56:06.355631 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb86dd0-ea57-41f5-a6ef-2e8b71782db2 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:06.355914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb86dd0-ea57-41f5-a6ef-2e8b71782db2 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:06.356184 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb86dd0-ea57-41f5-a6ef-2e8b71782db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:06.422639 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-9cb86dd0-ea57-41f5-a6ef-2e8b71782db2 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:56:06.423989 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-9cb86dd0-ea57-41f5-a6ef-2e8b71782db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:06.522617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 686/1375] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:56:06.819263 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e3318-857c-4689-877f-254c07f0a963 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:56:06.820853 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-690e3318-857c-4689-877f-254c07f0a963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:06.971542 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e3318-857c-4689-877f-254c07f0a963 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:56:07.029660 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:07.094923 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f92f92-57f6-4ab3-b1a2-8ae7554ddcfe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:56:07.095667 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f92f92-57f6-4ab3-b1a2-8ae7554ddcfe service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:07.095977 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f92f92-57f6-4ab3-b1a2-8ae7554ddcfe service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:07.096390 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f92f92-57f6-4ab3-b1a2-8ae7554ddcfe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:07.098339 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a9f92f92-57f6-4ab3-b1a2-8ae7554ddcfe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:07.197860 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f92f92-57f6-4ab3-b1a2-8ae7554ddcfe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:56:07.199546 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f92f92-57f6-4ab3-b1a2-8ae7554ddcfe service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:56:07.218272 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 690/1376] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:56:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:56:09.334704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470c2a3-e955-4dfb-8958-5666b84647d3 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:09.335591 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470c2a3-e955-4dfb-8958-5666b84647d3 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:09.335867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470c2a3-e955-4dfb-8958-5666b84647d3 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:09.337218 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 687/1377] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:56:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:56:09.342646 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fc82f-58ab-46db-ad20-44559d3fe9ee None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:09.343103 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fc82f-58ab-46db-ad20-44559d3fe9ee None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:09.343224 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fc82f-58ab-46db-ad20-44559d3fe9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:09.415465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-6a1fc82f-58ab-46db-ad20-44559d3fe9ee None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:56:09.416676 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-6a1fc82f-58ab-46db-ad20-44559d3fe9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:09.424085 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:09.538875 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 691/1378] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:56:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:56:09.835428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f470c2a3-e955-4dfb-8958-5666b84647d3 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:56:09.837495 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-f470c2a3-e955-4dfb-8958-5666b84647d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:09.910506 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:10.001620 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f470c2a3-e955-4dfb-8958-5666b84647d3 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:56:10.130795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccc001-bf2b-4aeb-bc0f-82ce967ef533 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:56:10.131410 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccc001-bf2b-4aeb-bc0f-82ce967ef533 service nova] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:10.131692 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccc001-bf2b-4aeb-bc0f-82ce967ef533 service nova] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:10.131967 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccc001-bf2b-4aeb-bc0f-82ce967ef533 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:10.133607 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-57ccc001-bf2b-4aeb-bc0f-82ce967ef533 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:10.153617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:10.234412 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ccc001-bf2b-4aeb-bc0f-82ce967ef533 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:56:10.236044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ccc001-bf2b-4aeb-bc0f-82ce967ef533 service nova] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:56:10.255610 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 688/1379] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:56:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:56:12.475553 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abea11-ce6b-4cc1-845e-35bae99a72c3 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:12.476199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abea11-ce6b-4cc1-845e-35bae99a72c3 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:12.476199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abea11-ce6b-4cc1-845e-35bae99a72c3 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:12.477783 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 692/1380] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:56:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:56:12.485157 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca6e54-3051-48c1-802e-4c67a132583f None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:12.485743 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca6e54-3051-48c1-802e-4c67a132583f None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:12.485743 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca6e54-3051-48c1-802e-4c67a132583f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:12.563368 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-5aca6e54-3051-48c1-802e-4c67a132583f None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:56:12.564942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-5aca6e54-3051-48c1-802e-4c67a132583f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:12.678557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:12.686428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 689/1381] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:56:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:56:12.991161 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93abea11-ce6b-4cc1-845e-35bae99a72c3 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:56:12.992793 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-93abea11-ce6b-4cc1-845e-35bae99a72c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:13.152229 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:13.163400 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93abea11-ce6b-4cc1-845e-35bae99a72c3 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:56:13.288470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae0e82-c226-4659-8b5f-759acfcef1c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:56:13.288952 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae0e82-c226-4659-8b5f-759acfcef1c0 service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:13.289239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae0e82-c226-4659-8b5f-759acfcef1c0 service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:13.289573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae0e82-c226-4659-8b5f-759acfcef1c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:13.291134 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-daae0e82-c226-4659-8b5f-759acfcef1c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:13.386138 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae0e82-c226-4659-8b5f-759acfcef1c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:56:13.392486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae0e82-c226-4659-8b5f-759acfcef1c0 service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:56:13.398098 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:13.408000 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 693/1382] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:56:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:56:15.641573 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292624b7-fc3d-450e-bbe2-66633153bd3e None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:15.642194 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292624b7-fc3d-450e-bbe2-66633153bd3e None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:15.642194 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292624b7-fc3d-450e-bbe2-66633153bd3e None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:15.643633 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 690/1383] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:56:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:56:15.649198 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab109af-c726-40d0-88ec-94f051924d83 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:15.649642 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab109af-c726-40d0-88ec-94f051924d83 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:15.649905 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab109af-c726-40d0-88ec-94f051924d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:15.718244 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-8ab109af-c726-40d0-88ec-94f051924d83 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:56:15.719495 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-8ab109af-c726-40d0-88ec-94f051924d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:15.820442 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 694/1384] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:56:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:56:16.193075 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292624b7-fc3d-450e-bbe2-66633153bd3e None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:56:16.194672 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-292624b7-fc3d-450e-bbe2-66633153bd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:16.372501 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292624b7-fc3d-450e-bbe2-66633153bd3e None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:56:16.459540 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:16.515090 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789dbf1f-4d74-4685-a8c0-6524487ca5ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:56:16.515709 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789dbf1f-4d74-4685-a8c0-6524487ca5ec service nova] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:16.515987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789dbf1f-4d74-4685-a8c0-6524487ca5ec service nova] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:16.516263 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789dbf1f-4d74-4685-a8c0-6524487ca5ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:16.517974 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-789dbf1f-4d74-4685-a8c0-6524487ca5ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:16.664936 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789dbf1f-4d74-4685-a8c0-6524487ca5ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:56:16.667113 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789dbf1f-4d74-4685-a8c0-6524487ca5ec service nova] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:56:16.699897 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 691/1385] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:56:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:56:19.182238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab1034-a2b9-4725-bc0d-dcf376990c82 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:19.182881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab1034-a2b9-4725-bc0d-dcf376990c82 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:19.183086 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab1034-a2b9-4725-bc0d-dcf376990c82 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:19.184786 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 695/1386] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:56:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:56:19.190756 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c28d90-824c-48b2-8164-088dca8c8284 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:19.191209 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c28d90-824c-48b2-8164-088dca8c8284 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:19.191384 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c28d90-824c-48b2-8164-088dca8c8284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:19.278603 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-f4c28d90-824c-48b2-8164-088dca8c8284 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:56:19.280234 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-f4c28d90-824c-48b2-8164-088dca8c8284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:19.316375 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:19.383542 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 692/1387] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:56:19.683025 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab1034-a2b9-4725-bc0d-dcf376990c82 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:56:19.685046 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-c1ab1034-a2b9-4725-bc0d-dcf376990c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:19.764046 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:19.836310 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab1034-a2b9-4725-bc0d-dcf376990c82 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:56:19.958584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3041b4f3-f23b-457a-94c2-9e0600f188cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:56:19.958584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041b4f3-f23b-457a-94c2-9e0600f188cb service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:19.959661 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041b4f3-f23b-457a-94c2-9e0600f188cb service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:19.959954 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041b4f3-f23b-457a-94c2-9e0600f188cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:19.961790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-3041b4f3-f23b-457a-94c2-9e0600f188cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:20.006560 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:20.061130 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3041b4f3-f23b-457a-94c2-9e0600f188cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:56:20.062793 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3041b4f3-f23b-457a-94c2-9e0600f188cb service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:56:20.081957 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 696/1388] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:56:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:56:21.461209 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eeb5a7-c837-4491-8cad-abd19b3e431d None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:21.461784 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eeb5a7-c837-4491-8cad-abd19b3e431d None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:21.461784 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eeb5a7-c837-4491-8cad-abd19b3e431d None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:21.463140 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 693/1389] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:56:21.469336 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70383489-2798-4e84-a5e9-8ee48ac2db30 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:21.469807 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70383489-2798-4e84-a5e9-8ee48ac2db30 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:21.469900 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70383489-2798-4e84-a5e9-8ee48ac2db30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:21.537306 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-70383489-2798-4e84-a5e9-8ee48ac2db30 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:56:21.538559 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-70383489-2798-4e84-a5e9-8ee48ac2db30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:21.637563 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 697/1390] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:56:21.946679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e73b31-9ca5-45c6-b2da-07b6dc4f14e1 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:21.946955 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e73b31-9ca5-45c6-b2da-07b6dc4f14e1 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:21.947225 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e73b31-9ca5-45c6-b2da-07b6dc4f14e1 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:21.949904 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 694/1391] 10.222.0.16 () {54 vars in 967 bytes} [Wed Aug 31 10:56:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:56:21.954905 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390940d9-8e6f-4207-b485-9c8fbb92a556 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:21.955193 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390940d9-8e6f-4207-b485-9c8fbb92a556 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:21.955466 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390940d9-8e6f-4207-b485-9c8fbb92a556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:21.972158 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:22.038432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-390940d9-8e6f-4207-b485-9c8fbb92a556 None None] MFA Rules not processed for user `f2e72c8e7b444e978bf99c6404febcd2`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:56:22.039646 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-390940d9-8e6f-4207-b485-9c8fbb92a556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:22.123423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 698/1392] 10.222.0.16 () {58 vars in 1064 bytes} [Wed Aug 31 10:56:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:56:22.131399 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e73b31-9ca5-45c6-b2da-07b6dc4f14e1 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:56:22.132906 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-03e73b31-9ca5-45c6-b2da-07b6dc4f14e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:22.148195 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339b1fc-40f5-4f70-8602-723d61a15dd9 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:22.148611 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339b1fc-40f5-4f70-8602-723d61a15dd9 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:22.149037 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339b1fc-40f5-4f70-8602-723d61a15dd9 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:22.150474 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 699/1393] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:56:22.156403 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac2439-8299-47e0-b147-c97c3c220b2d None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:22.156676 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac2439-8299-47e0-b147-c97c3c220b2d None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:22.156943 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac2439-8299-47e0-b147-c97c3c220b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:22.226264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-b8ac2439-8299-47e0-b147-c97c3c220b2d None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:56:22.227488 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-b8ac2439-8299-47e0-b147-c97c3c220b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:22.244585 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e73b31-9ca5-45c6-b2da-07b6dc4f14e1 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:56:22.287293 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:22.310211 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:22.327576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17dc71-3084-46c7-bc08-d63e2b49b6c5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2e72c8e7b444e978bf99c6404febcd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:56:22.328316 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17dc71-3084-46c7-bc08-d63e2b49b6c5 service glance] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:22.328713 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17dc71-3084-46c7-bc08-d63e2b49b6c5 service glance] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:22.328985 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17dc71-3084-46c7-bc08-d63e2b49b6c5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:22.330652 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-5a17dc71-3084-46c7-bc08-d63e2b49b6c5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:22.332449 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 700/1394] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:56:22.444978 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17dc71-3084-46c7-bc08-d63e2b49b6c5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:56:22.446478 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17dc71-3084-46c7-bc08-d63e2b49b6c5 service glance] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:56:22.466121 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 695/1395] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:56:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 10:56:22.621552 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac2439-8299-47e0-b147-c97c3c220b2d None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:56:22.622916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-b8ac2439-8299-47e0-b147-c97c3c220b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:22.752844 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac2439-8299-47e0-b147-c97c3c220b2d None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:56:22.824252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:22.873169 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67538a0-6e8c-4bed-bc2c-fd175c717ac2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:56:22.873705 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67538a0-6e8c-4bed-bc2c-fd175c717ac2 service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:22.874016 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67538a0-6e8c-4bed-bc2c-fd175c717ac2 service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:22.874287 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67538a0-6e8c-4bed-bc2c-fd175c717ac2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:22.875938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-f67538a0-6e8c-4bed-bc2c-fd175c717ac2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:22.976139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67538a0-6e8c-4bed-bc2c-fd175c717ac2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:56:22.977738 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67538a0-6e8c-4bed-bc2c-fd175c717ac2 service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:56:22.996875 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 701/1396] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:56:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:56:23.374662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97df30-10da-4b78-882c-1d72163fb778 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:23.375052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97df30-10da-4b78-882c-1d72163fb778 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:23.375394 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97df30-10da-4b78-882c-1d72163fb778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:23.446155 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-cd97df30-10da-4b78-882c-1d72163fb778 None None] MFA Rules not processed for user `7796ebff70054de090e02c58bf7000f4`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:56:23.447329 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-cd97df30-10da-4b78-882c-1d72163fb778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:23.489323 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:23.538577 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 696/1397] 10.222.0.16 () {58 vars in 1009 bytes} [Wed Aug 31 10:56:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:56:23.595670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b34e51-9505-4de5-bb59-ea8453436797 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:23.596088 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b34e51-9505-4de5-bb59-ea8453436797 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:23.596539 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b34e51-9505-4de5-bb59-ea8453436797 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:23.598333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 702/1398] 10.222.0.16 () {54 vars in 943 bytes} [Wed Aug 31 10:56:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:56:23.606813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7ab70-8530-428b-b3d8-681482d6724f None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:23.607393 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7ab70-8530-428b-b3d8-681482d6724f None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:23.607650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7ab70-8530-428b-b3d8-681482d6724f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:23.690897 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-6cb7ab70-8530-428b-b3d8-681482d6724f None None] MFA Rules not processed for user `e8abc7c439a04622896b34ed0a7b8492`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:56:23.692150 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-6cb7ab70-8530-428b-b3d8-681482d6724f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:23.802513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 697/1399] 10.222.0.16 () {58 vars in 1040 bytes} [Wed Aug 31 10:56:23 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:56:23.813399 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b34e51-9505-4de5-bb59-ea8453436797 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:56:23.815036 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-94b34e51-9505-4de5-bb59-ea8453436797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:23.899367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:23.958093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b34e51-9505-4de5-bb59-ea8453436797 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:56:24.066153 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb7192-1e5e-4d3c-9547-07e47cee1d4f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8abc7c439a04622896b34ed0a7b8492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:56:24.066701 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb7192-1e5e-4d3c-9547-07e47cee1d4f service swift] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:24.066981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb7192-1e5e-4d3c-9547-07e47cee1d4f service swift] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:24.067264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb7192-1e5e-4d3c-9547-07e47cee1d4f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:24.068928 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-73bb7192-1e5e-4d3c-9547-07e47cee1d4f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:24.143469 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:24.163213 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb7192-1e5e-4d3c-9547-07e47cee1d4f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:56:24.170978 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb7192-1e5e-4d3c-9547-07e47cee1d4f service swift] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:56:24.174065 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 703/1400] 10.222.0.16 () {60 vars in 1384 bytes} [Wed Aug 31 10:56:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 10:56:25.075547 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7d578-b4c8-46eb-bf54-63a8e76e072a None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:25.076254 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7d578-b4c8-46eb-bf54-63a8e76e072a None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:25.076254 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7d578-b4c8-46eb-bf54-63a8e76e072a None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:25.077654 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 698/1401] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:56:25.084025 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24bfbc-6b7f-4958-90ce-e7bdabbe09ca None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:25.084025 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24bfbc-6b7f-4958-90ce-e7bdabbe09ca None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:25.084025 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24bfbc-6b7f-4958-90ce-e7bdabbe09ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:25.170710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-3d24bfbc-6b7f-4958-90ce-e7bdabbe09ca None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:56:25.171976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-3d24bfbc-6b7f-4958-90ce-e7bdabbe09ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:25.293623 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 704/1402] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:56:25.593689 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf7d578-b4c8-46eb-bf54-63a8e76e072a None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:56:25.595272 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-caf7d578-b4c8-46eb-bf54-63a8e76e072a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:25.611389 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:25.765343 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf7d578-b4c8-46eb-bf54-63a8e76e072a None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:56:25.865053 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:25.901359 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1539b7c-c6e5-4681-a474-bb8f645125a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:56:25.902675 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1539b7c-c6e5-4681-a474-bb8f645125a0 service nova] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:25.903043 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1539b7c-c6e5-4681-a474-bb8f645125a0 service nova] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:25.903434 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1539b7c-c6e5-4681-a474-bb8f645125a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:25.905118 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-b1539b7c-c6e5-4681-a474-bb8f645125a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:26.016142 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1539b7c-c6e5-4681-a474-bb8f645125a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:56:26.017797 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1539b7c-c6e5-4681-a474-bb8f645125a0 service nova] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:56:26.043465 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 699/1403] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:56:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:56:28.198571 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51e1be-ed8a-40d2-b607-d2b855b45d6d None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:28.199142 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51e1be-ed8a-40d2-b607-d2b855b45d6d None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:28.199142 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51e1be-ed8a-40d2-b607-d2b855b45d6d None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:28.202859 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 705/1404] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:56:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:56:28.209333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ad2e0-0872-4437-83a3-a89ee7b012c9 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:28.209914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ad2e0-0872-4437-83a3-a89ee7b012c9 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:28.210118 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ad2e0-0872-4437-83a3-a89ee7b012c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:28.283290 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-870ad2e0-0872-4437-83a3-a89ee7b012c9 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:56:28.285447 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-870ad2e0-0872-4437-83a3-a89ee7b012c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:28.343268 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:28.397600 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 700/1405] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:56:28.704791 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f51e1be-ed8a-40d2-b607-d2b855b45d6d None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:56:28.706437 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-4f51e1be-ed8a-40d2-b607-d2b855b45d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:28.907395 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:29.044385 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f51e1be-ed8a-40d2-b607-d2b855b45d6d None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:56:29.182269 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640afd1-8fb0-4aa4-9684-d58ffd2f928a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:56:29.183364 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640afd1-8fb0-4aa4-9684-d58ffd2f928a service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:29.183771 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640afd1-8fb0-4aa4-9684-d58ffd2f928a service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:29.184044 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640afd1-8fb0-4aa4-9684-d58ffd2f928a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:29.185662 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-e640afd1-8fb0-4aa4-9684-d58ffd2f928a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:29.206534 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:29.287122 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640afd1-8fb0-4aa4-9684-d58ffd2f928a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:56:29.288664 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640afd1-8fb0-4aa4-9684-d58ffd2f928a service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:56:29.313161 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 706/1406] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:56:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:56:31.526449 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be27573-2a4f-4575-91ec-43fd3f9dfa5d None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:31.527122 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be27573-2a4f-4575-91ec-43fd3f9dfa5d None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:31.527122 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be27573-2a4f-4575-91ec-43fd3f9dfa5d None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:31.528616 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 701/1407] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:56:31.533940 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cbdb65-d977-42b4-aced-8ee2e5d10900 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:31.534794 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cbdb65-d977-42b4-aced-8ee2e5d10900 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:31.534794 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cbdb65-d977-42b4-aced-8ee2e5d10900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:31.608688 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-b1cbdb65-d977-42b4-aced-8ee2e5d10900 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:56:31.609956 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-b1cbdb65-d977-42b4-aced-8ee2e5d10900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:31.711221 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:31.717976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 707/1408] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:56:32.010765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be27573-2a4f-4575-91ec-43fd3f9dfa5d None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:56:32.012448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-5be27573-2a4f-4575-91ec-43fd3f9dfa5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:32.185347 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be27573-2a4f-4575-91ec-43fd3f9dfa5d None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:56:32.221298 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:32.319402 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b068c-e070-455d-9cae-0272c2889e29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:56:32.320037 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b068c-e070-455d-9cae-0272c2889e29 service nova] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:32.320375 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b068c-e070-455d-9cae-0272c2889e29 service nova] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:32.320704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b068c-e070-455d-9cae-0272c2889e29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:32.322409 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-606b068c-e070-455d-9cae-0272c2889e29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:32.438595 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606b068c-e070-455d-9cae-0272c2889e29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:56:32.440193 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606b068c-e070-455d-9cae-0272c2889e29 service nova] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:56:32.465135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 702/1409] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:56:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:56:41.746319 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=2 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:42.462255 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=83 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:48.196072 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e6905-68bd-479e-a567-59a31d388340 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:48.196604 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e6905-68bd-479e-a567-59a31d388340 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:48.196714 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e6905-68bd-479e-a567-59a31d388340 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:48.198016 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 708/1410] 10.222.0.16 () {54 vars in 918 bytes} [Wed Aug 31 10:56:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:56:48.203354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed560412-7200-4e74-8b3d-da9d126eb2ff None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:48.203793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed560412-7200-4e74-8b3d-da9d126eb2ff None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:48.203934 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed560412-7200-4e74-8b3d-da9d126eb2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:48.286670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-ed560412-7200-4e74-8b3d-da9d126eb2ff None None] MFA Rules not processed for user `f3db46a4b4e74cc0bb2a7bd6f0e99105`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:56:48.287938 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-ed560412-7200-4e74-8b3d-da9d126eb2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:48.404679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 703/1411] 10.222.0.16 () {58 vars in 1015 bytes} [Wed Aug 31 10:56:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:56:48.420304 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe68b8-b973-4b60-8ad8-6711b26ad3ec None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:48.420647 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe68b8-b973-4b60-8ad8-6711b26ad3ec None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:48.420841 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe68b8-b973-4b60-8ad8-6711b26ad3ec None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:48.422333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 709/1412] 10.222.0.16 () {54 vars in 918 bytes} [Wed Aug 31 10:56:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:56:48.427349 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f073d-3d96-45ec-bc62-fa9eb1788be9 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:48.427647 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f073d-3d96-45ec-bc62-fa9eb1788be9 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:48.427927 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f073d-3d96-45ec-bc62-fa9eb1788be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:48.509468 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:48.512950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-f02f073d-3d96-45ec-bc62-fa9eb1788be9 None None] MFA Rules not processed for user `f11971a594014d86968c3356d11d42f7`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:56:48.514163 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-f02f073d-3d96-45ec-bc62-fa9eb1788be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:48.651363 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 704/1413] 10.222.0.16 () {58 vars in 1015 bytes} [Wed Aug 31 10:56:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:56:51.050972 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57661e2e-5453-4285-bbaf-020bfc8dfe7b None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:51.051532 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57661e2e-5453-4285-bbaf-020bfc8dfe7b None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:51.051532 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57661e2e-5453-4285-bbaf-020bfc8dfe7b None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:51.052761 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 10:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:56:51.058749 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24d25d-460c-44a5-8465-1d578800fdf9 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:51.059163 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24d25d-460c-44a5-8465-1d578800fdf9 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:51.059279 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24d25d-460c-44a5-8465-1d578800fdf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:51.125705 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-bf24d25d-460c-44a5-8465-1d578800fdf9 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:56:51.126933 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-bf24d25d-460c-44a5-8465-1d578800fdf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:51.181094 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:51.222548 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 10:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:56:51.514010 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57661e2e-5453-4285-bbaf-020bfc8dfe7b None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:56:51.515542 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-57661e2e-5453-4285-bbaf-020bfc8dfe7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:51.659439 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57661e2e-5453-4285-bbaf-020bfc8dfe7b None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:56:51.732986 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:51.771962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b69f57-a65a-4aed-8d64-86df17233a3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:56:51.772558 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b69f57-a65a-4aed-8d64-86df17233a3c service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:51.772829 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b69f57-a65a-4aed-8d64-86df17233a3c service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:51.773093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b69f57-a65a-4aed-8d64-86df17233a3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:51.774704 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-72b69f57-a65a-4aed-8d64-86df17233a3c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:51.862074 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b69f57-a65a-4aed-8d64-86df17233a3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:56:51.863734 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b69f57-a65a-4aed-8d64-86df17233a3c service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:56:51.881363 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 711/1416] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:56:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:56:52.814566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26c323-1378-4975-bd5a-c07915cd2be8 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:52.815092 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26c323-1378-4975-bd5a-c07915cd2be8 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:52.815092 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26c323-1378-4975-bd5a-c07915cd2be8 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:52.816460 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 10:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:56:52.821502 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854dcf06-67f8-4afa-9589-549aa0f17dc9 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:52.821913 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854dcf06-67f8-4afa-9589-549aa0f17dc9 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:52.822085 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854dcf06-67f8-4afa-9589-549aa0f17dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:52.897211 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-854dcf06-67f8-4afa-9589-549aa0f17dc9 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:56:52.898462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-854dcf06-67f8-4afa-9589-549aa0f17dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:52.926650 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:53.001150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 10:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:56:53.163380 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26c323-1378-4975-bd5a-c07915cd2be8 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:56:53.164890 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-ed26c323-1378-4975-bd5a-c07915cd2be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:53.317006 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26c323-1378-4975-bd5a-c07915cd2be8 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:56:53.355789 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:53.430511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1e836-057a-4a75-8d84-2b19db5218dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:56:53.431125 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1e836-057a-4a75-8d84-2b19db5218dd service nova] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:53.431402 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1e836-057a-4a75-8d84-2b19db5218dd service nova] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:53.431670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1e836-057a-4a75-8d84-2b19db5218dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:53.434004 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-f5a1e836-057a-4a75-8d84-2b19db5218dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:53.523484 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a1e836-057a-4a75-8d84-2b19db5218dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:56:53.525075 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a1e836-057a-4a75-8d84-2b19db5218dd service nova] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:56:53.545316 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 707/1419] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:56:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:56:54.449948 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac2c6d-209e-4d2b-912f-294462e30a7f None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:54.450582 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac2c6d-209e-4d2b-912f-294462e30a7f None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:54.450582 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac2c6d-209e-4d2b-912f-294462e30a7f None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:54.451861 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 10:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:56:54.456926 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec6d4d-aadc-4781-8691-3fcde2f70065 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:54.457365 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec6d4d-aadc-4781-8691-3fcde2f70065 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:54.457472 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec6d4d-aadc-4781-8691-3fcde2f70065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:54.631818 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:54.653895 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-e0ec6d4d-aadc-4781-8691-3fcde2f70065 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:56:54.655145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-e0ec6d4d-aadc-4781-8691-3fcde2f70065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:54.756730 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 10:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:56:54.906959 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac2c6d-209e-4d2b-912f-294462e30a7f None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:56:54.908531 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-f9ac2c6d-209e-4d2b-912f-294462e30a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:55.053539 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac2c6d-209e-4d2b-912f-294462e30a7f None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:56:55.066343 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:55.167175 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5908b0-7e12-4c20-a116-a7df752e68f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:56:55.167778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5908b0-7e12-4c20-a116-a7df752e68f6 service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:55.168053 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5908b0-7e12-4c20-a116-a7df752e68f6 service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:55.168321 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5908b0-7e12-4c20-a116-a7df752e68f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:55.169918 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-5b5908b0-7e12-4c20-a116-a7df752e68f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:55.259104 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5908b0-7e12-4c20-a116-a7df752e68f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:56:55.261545 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5908b0-7e12-4c20-a116-a7df752e68f6 service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:56:55.280437 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 714/1422] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:56:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:56:56.174221 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffc655-9566-4d2c-97a1-9f434a5d5d1f None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:56.174826 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffc655-9566-4d2c-97a1-9f434a5d5d1f None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:56.174826 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffc655-9566-4d2c-97a1-9f434a5d5d1f None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:56.176278 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 10:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:56:56.181101 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b7c34-11d7-48a9-a697-77286a6323a4 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:56.181545 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b7c34-11d7-48a9-a697-77286a6323a4 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:56.181735 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b7c34-11d7-48a9-a697-77286a6323a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:56.202358 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:56.259985 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-a71b7c34-11d7-48a9-a697-77286a6323a4 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:56:56.261845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a71b7c34-11d7-48a9-a697-77286a6323a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:56.357826 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 10:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:56:56.508356 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ffc655-9566-4d2c-97a1-9f434a5d5d1f None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:56:56.509861 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a7ffc655-9566-4d2c-97a1-9f434a5d5d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:56.618140 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:56.653293 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ffc655-9566-4d2c-97a1-9f434a5d5d1f None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:56:56.769917 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ad458-26a8-48cc-90cd-b59ac9edacd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:56:56.770568 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ad458-26a8-48cc-90cd-b59ac9edacd5 service nova] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:56.770903 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ad458-26a8-48cc-90cd-b59ac9edacd5 service nova] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:56.771172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ad458-26a8-48cc-90cd-b59ac9edacd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:56.772953 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a64ad458-26a8-48cc-90cd-b59ac9edacd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:56.856149 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:56.871407 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ad458-26a8-48cc-90cd-b59ac9edacd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:56:56.871989 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ad458-26a8-48cc-90cd-b59ac9edacd5 service nova] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:56:56.890223 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 710/1425] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:56:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:56:57.783102 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d5698-c107-4ba8-99f3-e0c7bbf62948 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:57.783708 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d5698-c107-4ba8-99f3-e0c7bbf62948 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:57.783708 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d5698-c107-4ba8-99f3-e0c7bbf62948 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:57.784995 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 10:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:56:57.790222 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72dbd2-1a28-4b31-9108-2f6776ea0e8e None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:57.793475 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72dbd2-1a28-4b31-9108-2f6776ea0e8e None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:57.793475 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72dbd2-1a28-4b31-9108-2f6776ea0e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:57.866324 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-5e72dbd2-1a28-4b31-9108-2f6776ea0e8e None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:56:57.867553 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-5e72dbd2-1a28-4b31-9108-2f6776ea0e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:57.962951 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 10:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:56:58.113442 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d5698-c107-4ba8-99f3-e0c7bbf62948 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:56:58.117256 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a21d5698-c107-4ba8-99f3-e0c7bbf62948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:58.199000 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:58.260482 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d5698-c107-4ba8-99f3-e0c7bbf62948 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:56:58.374154 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a3bcf-c313-42b6-b64b-4f29d9a87b50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:56:58.374779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a3bcf-c313-42b6-b64b-4f29d9a87b50 service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:58.375052 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a3bcf-c313-42b6-b64b-4f29d9a87b50 service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:58.375316 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a3bcf-c313-42b6-b64b-4f29d9a87b50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:58.376894 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-8f5a3bcf-c313-42b6-b64b-4f29d9a87b50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:58.425720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:58.467842 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a3bcf-c313-42b6-b64b-4f29d9a87b50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:56:58.469439 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a3bcf-c313-42b6-b64b-4f29d9a87b50 service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:56:58.488777 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 717/1428] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:56:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:56:59.404657 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f1815-e713-4e09-8802-51dd3e7fdd93 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:59.405195 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f1815-e713-4e09-8802-51dd3e7fdd93 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:59.405195 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f1815-e713-4e09-8802-51dd3e7fdd93 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:59.406503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 10:56:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:56:59.413214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48974c96-1dc9-4620-bb2d-053008edd9dc None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:56:59.413214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48974c96-1dc9-4620-bb2d-053008edd9dc None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:56:59.413214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48974c96-1dc9-4620-bb2d-053008edd9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:56:59.487746 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-48974c96-1dc9-4620-bb2d-053008edd9dc None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:56:59.488956 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-48974c96-1dc9-4620-bb2d-053008edd9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:59.587198 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 10:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:56:59.740311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f1815-e713-4e09-8802-51dd3e7fdd93 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:56:59.741790 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-413f1815-e713-4e09-8802-51dd3e7fdd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:56:59.790355 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:59.887579 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f1815-e713-4e09-8802-51dd3e7fdd93 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:00.001198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b122e-7c68-40b5-bd95-61e12c568cab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:00.001734 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b122e-7c68-40b5-bd95-61e12c568cab service nova] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:00.002010 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b122e-7c68-40b5-bd95-61e12c568cab service nova] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:00.002277 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b122e-7c68-40b5-bd95-61e12c568cab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:00.003932 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-298b122e-7c68-40b5-bd95-61e12c568cab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:00.011358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:00.093738 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b122e-7c68-40b5-bd95-61e12c568cab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:00.095464 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b122e-7c68-40b5-bd95-61e12c568cab service nova] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:00.114323 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 713/1431] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:56:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:57:00.977477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ea863-1a3e-46e0-80d6-b4ee083d6a6c None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:00.978004 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ea863-1a3e-46e0-80d6-b4ee083d6a6c None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:00.978004 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ea863-1a3e-46e0-80d6-b4ee083d6a6c None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:00.979909 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 10:57:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:00.985024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0668e0b-ec0d-4890-9376-407790a90cb0 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:00.985458 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0668e0b-ec0d-4890-9376-407790a90cb0 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:00.985559 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0668e0b-ec0d-4890-9376-407790a90cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:01.060182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-f0668e0b-ec0d-4890-9376-407790a90cb0 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:01.061387 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-f0668e0b-ec0d-4890-9376-407790a90cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:01.131106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:01.160196 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 10:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:01.314320 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ea863-1a3e-46e0-80d6-b4ee083d6a6c None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:57:01.315798 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-c43ea863-1a3e-46e0-80d6-b4ee083d6a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:01.333142 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:01.459288 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ea863-1a3e-46e0-80d6-b4ee083d6a6c None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:01.546347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:01.571703 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13641292-d12c-4228-8987-589603fe0193 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:01.572238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13641292-d12c-4228-8987-589603fe0193 service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:01.572646 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13641292-d12c-4228-8987-589603fe0193 service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:01.573038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13641292-d12c-4228-8987-589603fe0193 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:01.574671 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-13641292-d12c-4228-8987-589603fe0193 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:01.665150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13641292-d12c-4228-8987-589603fe0193 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:01.667191 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13641292-d12c-4228-8987-589603fe0193 service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:01.685313 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 720/1434] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:57:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:57:02.591027 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d002e55-8a9e-4e2b-b04f-09908183b193 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:02.591597 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d002e55-8a9e-4e2b-b04f-09908183b193 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:02.591597 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d002e55-8a9e-4e2b-b04f-09908183b193 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:02.592969 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 10:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:02.598359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a76b5-9423-48d6-8eba-ae40004f8d33 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:02.598842 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a76b5-9423-48d6-8eba-ae40004f8d33 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:02.598943 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a76b5-9423-48d6-8eba-ae40004f8d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:02.676037 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-d82a76b5-9423-48d6-8eba-ae40004f8d33 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:02.677234 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-d82a76b5-9423-48d6-8eba-ae40004f8d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:02.721400 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:02.777605 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 10:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:02.930711 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d002e55-8a9e-4e2b-b04f-09908183b193 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:57:02.932801 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-8d002e55-8a9e-4e2b-b04f-09908183b193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:03.077574 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d002e55-8a9e-4e2b-b04f-09908183b193 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:03.131692 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:03.191047 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608feac-66e6-4589-94fb-4e738179cbaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:03.191572 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608feac-66e6-4589-94fb-4e738179cbaa service nova] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:03.191845 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608feac-66e6-4589-94fb-4e738179cbaa service nova] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:03.192108 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608feac-66e6-4589-94fb-4e738179cbaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:03.193700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-0608feac-66e6-4589-94fb-4e738179cbaa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:03.280038 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608feac-66e6-4589-94fb-4e738179cbaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:03.281608 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608feac-66e6-4589-94fb-4e738179cbaa service nova] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:03.299762 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 716/1437] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:57:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:57:04.210873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465af6d-817c-4653-8a06-238d78a355e5 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:04.211401 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465af6d-817c-4653-8a06-238d78a355e5 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:04.211524 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465af6d-817c-4653-8a06-238d78a355e5 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:04.212885 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 10:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:04.218383 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6035a0c-b13a-4245-885c-9f94ce8dc81c None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:04.218808 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6035a0c-b13a-4245-885c-9f94ce8dc81c None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:04.218940 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6035a0c-b13a-4245-885c-9f94ce8dc81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:04.295261 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-a6035a0c-b13a-4245-885c-9f94ce8dc81c None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:04.296517 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a6035a0c-b13a-4245-885c-9f94ce8dc81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:04.298546 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:04.391503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 10:57:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:04.541652 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465af6d-817c-4653-8a06-238d78a355e5 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:57:04.543291 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-b465af6d-817c-4653-8a06-238d78a355e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:04.689091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465af6d-817c-4653-8a06-238d78a355e5 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:04.717647 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:04.802200 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374750b-df7c-406d-9c70-f276cacd9807 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:04.802827 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374750b-df7c-406d-9c70-f276cacd9807 service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:04.803105 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374750b-df7c-406d-9c70-f276cacd9807 service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:04.803373 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374750b-df7c-406d-9c70-f276cacd9807 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:04.804967 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-d374750b-df7c-406d-9c70-f276cacd9807 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:04.894834 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374750b-df7c-406d-9c70-f276cacd9807 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:04.896475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374750b-df7c-406d-9c70-f276cacd9807 service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:04.914873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 723/1440] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:57:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:57:06.776797 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eebf10-ba48-46fe-b463-1d39c20a9b96 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:06.777358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eebf10-ba48-46fe-b463-1d39c20a9b96 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:06.777358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eebf10-ba48-46fe-b463-1d39c20a9b96 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:06.778712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 10:57:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:06.784042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fabcc-5575-4d36-9e04-64f5b4b7e50e None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:06.784424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fabcc-5575-4d36-9e04-64f5b4b7e50e None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:06.784664 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fabcc-5575-4d36-9e04-64f5b4b7e50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:06.826573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:06.850799 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-437fabcc-5575-4d36-9e04-64f5b4b7e50e None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:06.857854 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-437fabcc-5575-4d36-9e04-64f5b4b7e50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:06.953650 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 10:57:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:07.243120 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eebf10-ba48-46fe-b463-1d39c20a9b96 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:57:07.245375 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-b2eebf10-ba48-46fe-b463-1d39c20a9b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:07.350032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eebf10-ba48-46fe-b463-1d39c20a9b96 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:07.382326 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:07.424498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f046116-d2f3-4a37-b8aa-3681b920c0d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:07.425396 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f046116-d2f3-4a37-b8aa-3681b920c0d1 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:07.425950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f046116-d2f3-4a37-b8aa-3681b920c0d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:07.426573 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f046116-d2f3-4a37-b8aa-3681b920c0d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:07.428254 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-2f046116-d2f3-4a37-b8aa-3681b920c0d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:07.538577 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f046116-d2f3-4a37-b8aa-3681b920c0d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:07.539922 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f046116-d2f3-4a37-b8aa-3681b920c0d1 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:07.559549 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 719/1443] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:57:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:57:09.642915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc095d1f-ecab-4e22-8632-11af32197a44 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:09.643509 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc095d1f-ecab-4e22-8632-11af32197a44 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:09.643509 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc095d1f-ecab-4e22-8632-11af32197a44 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:09.644825 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 10:57:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:09.650000 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659b0a8-23bc-49fe-a2e8-049cce88ea27 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:09.650586 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659b0a8-23bc-49fe-a2e8-049cce88ea27 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:09.650777 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659b0a8-23bc-49fe-a2e8-049cce88ea27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:09.717497 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-2659b0a8-23bc-49fe-a2e8-049cce88ea27 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:09.718744 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-2659b0a8-23bc-49fe-a2e8-049cce88ea27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:09.731476 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:09.816514 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 10:57:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:10.097494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc095d1f-ecab-4e22-8632-11af32197a44 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:57:10.099110 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-cc095d1f-ecab-4e22-8632-11af32197a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:10.202858 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc095d1f-ecab-4e22-8632-11af32197a44 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:10.214844 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:10.277972 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed01e84-0fd2-4777-89c2-552e7fae2266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:10.278600 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed01e84-0fd2-4777-89c2-552e7fae2266 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:10.278879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed01e84-0fd2-4777-89c2-552e7fae2266 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:10.279148 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed01e84-0fd2-4777-89c2-552e7fae2266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:10.280755 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-9ed01e84-0fd2-4777-89c2-552e7fae2266 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:10.388925 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed01e84-0fd2-4777-89c2-552e7fae2266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:10.390263 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed01e84-0fd2-4777-89c2-552e7fae2266 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:10.408606 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 726/1446] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:57:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:57:12.762754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab0500-a37f-49d9-be24-5d0f8bf15555 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:12.763309 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab0500-a37f-49d9-be24-5d0f8bf15555 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:12.763309 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab0500-a37f-49d9-be24-5d0f8bf15555 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:12.764805 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 10:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:12.770114 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f49b1-49bd-4174-a610-dd5df0e29ff6 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:12.770567 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f49b1-49bd-4174-a610-dd5df0e29ff6 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:12.770673 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f49b1-49bd-4174-a610-dd5df0e29ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:12.818458 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:12.837163 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-d80f49b1-49bd-4174-a610-dd5df0e29ff6 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:12.839131 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-d80f49b1-49bd-4174-a610-dd5df0e29ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:12.937199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 10:57:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:13.224723 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab0500-a37f-49d9-be24-5d0f8bf15555 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:57:13.226306 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-afab0500-a37f-49d9-be24-5d0f8bf15555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:13.340124 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab0500-a37f-49d9-be24-5d0f8bf15555 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:13.379867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:13.414286 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a478d5-b6c9-4db9-99b1-424a9d04af50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:13.415018 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a478d5-b6c9-4db9-99b1-424a9d04af50 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:13.415295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a478d5-b6c9-4db9-99b1-424a9d04af50 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:13.415603 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a478d5-b6c9-4db9-99b1-424a9d04af50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:13.418589 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-69a478d5-b6c9-4db9-99b1-424a9d04af50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:13.530752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a478d5-b6c9-4db9-99b1-424a9d04af50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:13.532191 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a478d5-b6c9-4db9-99b1-424a9d04af50 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:13.551231 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 722/1449] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:57:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:57:15.753024 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa946091-f31a-4c99-af68-af0a04c9ccc0 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:15.753612 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa946091-f31a-4c99-af68-af0a04c9ccc0 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:15.753612 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa946091-f31a-4c99-af68-af0a04c9ccc0 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:15.754932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 10:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:15.762578 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b54fc7-40cb-48d2-8e22-0e713f993892 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:15.763070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b54fc7-40cb-48d2-8e22-0e713f993892 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:15.763188 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b54fc7-40cb-48d2-8e22-0e713f993892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:15.829401 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-60b54fc7-40cb-48d2-8e22-0e713f993892 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:15.830647 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-60b54fc7-40cb-48d2-8e22-0e713f993892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:15.851067 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:15.932272 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 10:57:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:16.221005 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa946091-f31a-4c99-af68-af0a04c9ccc0 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:57:16.222626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-aa946091-f31a-4c99-af68-af0a04c9ccc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:16.327850 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa946091-f31a-4c99-af68-af0a04c9ccc0 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:16.348615 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:16.403828 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facdd961-9366-4d04-9061-d21657a73efd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:16.404351 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdd961-9366-4d04-9061-d21657a73efd service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:16.404631 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdd961-9366-4d04-9061-d21657a73efd service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:16.404899 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdd961-9366-4d04-9061-d21657a73efd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:16.406540 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-facdd961-9366-4d04-9061-d21657a73efd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:16.513860 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facdd961-9366-4d04-9061-d21657a73efd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:16.515311 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facdd961-9366-4d04-9061-d21657a73efd service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:16.533735 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 729/1452] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:57:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:57:18.599787 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae89c5b-1d5f-49a6-aede-e2cb304f4877 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:18.600373 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae89c5b-1d5f-49a6-aede-e2cb304f4877 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:18.600373 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae89c5b-1d5f-49a6-aede-e2cb304f4877 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:18.601485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 10:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:18.608490 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a0b72-0c7c-47d9-a9ba-9876b8ed2f6c None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:18.608896 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a0b72-0c7c-47d9-a9ba-9876b8ed2f6c None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:18.609024 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a0b72-0c7c-47d9-a9ba-9876b8ed2f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:18.673489 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:18.676368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-146a0b72-0c7c-47d9-a9ba-9876b8ed2f6c None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:18.677585 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-146a0b72-0c7c-47d9-a9ba-9876b8ed2f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:18.780095 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 10:57:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:19.068010 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae89c5b-1d5f-49a6-aede-e2cb304f4877 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:57:19.069565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-6ae89c5b-1d5f-49a6-aede-e2cb304f4877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:19.174129 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae89c5b-1d5f-49a6-aede-e2cb304f4877 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:19.222686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:19.247888 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8720a4d-180a-43da-8b00-8c5cd8572535 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:19.248488 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8720a4d-180a-43da-8b00-8c5cd8572535 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:19.248760 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8720a4d-180a-43da-8b00-8c5cd8572535 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:19.249025 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8720a4d-180a-43da-8b00-8c5cd8572535 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:19.250637 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-b8720a4d-180a-43da-8b00-8c5cd8572535 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:19.360350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8720a4d-180a-43da-8b00-8c5cd8572535 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:19.361715 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8720a4d-180a-43da-8b00-8c5cd8572535 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:19.381534 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 725/1455] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:57:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:57:21.416985 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8446e3-475f-4113-86f7-129d51bf89c1 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:21.417655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8446e3-475f-4113-86f7-129d51bf89c1 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:21.417655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8446e3-475f-4113-86f7-129d51bf89c1 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:21.418956 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 10:57:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:21.424332 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94707d62-2706-4fb3-9af4-5746a9e81dd8 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:21.424760 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94707d62-2706-4fb3-9af4-5746a9e81dd8 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:21.424880 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94707d62-2706-4fb3-9af4-5746a9e81dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:21.491058 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-94707d62-2706-4fb3-9af4-5746a9e81dd8 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:21.492303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-94707d62-2706-4fb3-9af4-5746a9e81dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:21.521459 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:21.592434 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 10:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:21.883598 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8446e3-475f-4113-86f7-129d51bf89c1 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:57:21.885642 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-7c8446e3-475f-4113-86f7-129d51bf89c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:21.991794 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8446e3-475f-4113-86f7-129d51bf89c1 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:22.020923 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:22.066504 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9508d-e067-440d-a6dd-aa6685d2ce8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:22.067101 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9508d-e067-440d-a6dd-aa6685d2ce8e service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:22.067400 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9508d-e067-440d-a6dd-aa6685d2ce8e service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:22.067669 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9508d-e067-440d-a6dd-aa6685d2ce8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:22.069264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-99b9508d-e067-440d-a6dd-aa6685d2ce8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:22.178179 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9508d-e067-440d-a6dd-aa6685d2ce8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:22.179648 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9508d-e067-440d-a6dd-aa6685d2ce8e service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:22.199359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 732/1458] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:57:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:57:24.191068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a552ca-773d-485a-ab0d-6d4ff128c471 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:24.191627 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a552ca-773d-485a-ab0d-6d4ff128c471 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:24.191627 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a552ca-773d-485a-ab0d-6d4ff128c471 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:24.192943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 10:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:24.199337 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b36424-2068-441e-a265-71a4a678eac0 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:24.199731 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b36424-2068-441e-a265-71a4a678eac0 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:24.199878 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b36424-2068-441e-a265-71a4a678eac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:24.265271 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-88b36424-2068-441e-a265-71a4a678eac0 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:24.266487 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-88b36424-2068-441e-a265-71a4a678eac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:24.274708 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:24.363852 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 10:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:24.648248 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a552ca-773d-485a-ab0d-6d4ff128c471 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:57:24.649921 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-00a552ca-773d-485a-ab0d-6d4ff128c471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:24.762286 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a552ca-773d-485a-ab0d-6d4ff128c471 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:24.828872 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:24.848472 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820566bb-993e-45db-a41f-54ef447209e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:24.849108 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820566bb-993e-45db-a41f-54ef447209e9 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:24.849382 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820566bb-993e-45db-a41f-54ef447209e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:24.849674 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820566bb-993e-45db-a41f-54ef447209e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:24.851309 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-820566bb-993e-45db-a41f-54ef447209e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:24.959132 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820566bb-993e-45db-a41f-54ef447209e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:24.960567 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820566bb-993e-45db-a41f-54ef447209e9 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:24.978177 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 728/1461] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:57:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:57:26.962155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097682f-498c-4d38-8452-ea8c8bc8df7f None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:26.965699 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097682f-498c-4d38-8452-ea8c8bc8df7f None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:26.965699 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097682f-498c-4d38-8452-ea8c8bc8df7f None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:26.965699 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 10:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:26.969667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e0b27-6dc4-46eb-ac10-34134dad121a None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:26.970085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e0b27-6dc4-46eb-ac10-34134dad121a None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:26.970266 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e0b27-6dc4-46eb-ac10-34134dad121a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:27.037753 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-381e0b27-6dc4-46eb-ac10-34134dad121a None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:27.038987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-381e0b27-6dc4-46eb-ac10-34134dad121a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:27.078239 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:27.137035 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 10:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:27.423220 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097682f-498c-4d38-8452-ea8c8bc8df7f None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:57:27.424859 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-5097682f-498c-4d38-8452-ea8c8bc8df7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:27.536670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097682f-498c-4d38-8452-ea8c8bc8df7f None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:27.574456 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:27.611486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4625583f-754d-489b-8def-7222871ce08c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:27.612093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625583f-754d-489b-8def-7222871ce08c service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:27.612373 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625583f-754d-489b-8def-7222871ce08c service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:27.612646 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625583f-754d-489b-8def-7222871ce08c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:27.614275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-4625583f-754d-489b-8def-7222871ce08c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:27.722264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4625583f-754d-489b-8def-7222871ce08c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:27.723726 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4625583f-754d-489b-8def-7222871ce08c service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:27.742613 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 735/1464] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:57:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:57:29.716440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342d4e1-af9c-4690-ba9d-59e89e464607 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:29.717040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342d4e1-af9c-4690-ba9d-59e89e464607 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:29.717040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342d4e1-af9c-4690-ba9d-59e89e464607 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:29.718317 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 10:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:29.723707 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a515c78-47ac-4e3e-af71-93a406ee83fd None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:29.724160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a515c78-47ac-4e3e-af71-93a406ee83fd None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:29.724289 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a515c78-47ac-4e3e-af71-93a406ee83fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:29.789668 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-2a515c78-47ac-4e3e-af71-93a406ee83fd None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:29.790887 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2a515c78-47ac-4e3e-af71-93a406ee83fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:29.811598 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:29.890558 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 10:57:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:30.172052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342d4e1-af9c-4690-ba9d-59e89e464607 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:57:30.173667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-d342d4e1-af9c-4690-ba9d-59e89e464607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:30.276354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342d4e1-af9c-4690-ba9d-59e89e464607 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:30.343074 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:30.349824 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f1553b-165c-4de6-b753-63510c7a0cdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:30.356415 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1553b-165c-4de6-b753-63510c7a0cdd service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:30.356415 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1553b-165c-4de6-b753-63510c7a0cdd service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:30.356415 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1553b-165c-4de6-b753-63510c7a0cdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:30.356415 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-79f1553b-165c-4de6-b753-63510c7a0cdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:30.461414 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f1553b-165c-4de6-b753-63510c7a0cdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:30.462874 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f1553b-165c-4de6-b753-63510c7a0cdd service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:30.481010 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 731/1467] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:57:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:57:32.406998 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61157e9f-6814-4f17-8bc2-b0b2488da72e None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:32.407558 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61157e9f-6814-4f17-8bc2-b0b2488da72e None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:32.407788 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61157e9f-6814-4f17-8bc2-b0b2488da72e None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:32.409191 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 10:57:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:32.417539 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebdaffc-16c0-48f1-ae34-cdd9aed1739f None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:32.424715 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebdaffc-16c0-48f1-ae34-cdd9aed1739f None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:32.424715 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebdaffc-16c0-48f1-ae34-cdd9aed1739f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:32.483735 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-eebdaffc-16c0-48f1-ae34-cdd9aed1739f None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:32.484937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-eebdaffc-16c0-48f1-ae34-cdd9aed1739f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:32.531956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:32.581304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 10:57:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:32.873226 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61157e9f-6814-4f17-8bc2-b0b2488da72e None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:57:32.874851 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-61157e9f-6814-4f17-8bc2-b0b2488da72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:32.977146 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61157e9f-6814-4f17-8bc2-b0b2488da72e None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:33.030756 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:33.055981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812754c-7e10-4e99-b26f-9501809c896f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:33.056587 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812754c-7e10-4e99-b26f-9501809c896f service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:33.056882 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812754c-7e10-4e99-b26f-9501809c896f service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:33.057149 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812754c-7e10-4e99-b26f-9501809c896f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:33.059183 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-f812754c-7e10-4e99-b26f-9501809c896f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:33.170953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f812754c-7e10-4e99-b26f-9501809c896f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:33.172390 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f812754c-7e10-4e99-b26f-9501809c896f service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:33.190163 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 738/1470] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:57:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:57:42.579840 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=22 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:43.187817 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=66 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:52.850654 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550f4ef-7714-4361-87a1-c31698f3bf51 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:52.851206 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550f4ef-7714-4361-87a1-c31698f3bf51 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:52.851206 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550f4ef-7714-4361-87a1-c31698f3bf51 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:52.852554 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 733/1471] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:52.858228 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6e53f-9def-4577-a2a5-193afc82c3b3 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:52.858671 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6e53f-9def-4577-a2a5-193afc82c3b3 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:52.858772 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6e53f-9def-4577-a2a5-193afc82c3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:52.923588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-61d6e53f-9def-4577-a2a5-193afc82c3b3 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:52.924784 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-61d6e53f-9def-4577-a2a5-193afc82c3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:53.024013 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 739/1472] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:57:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:53.337789 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504da90-fa1e-4132-bb1a-fcf610b1d2fb None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:53.338180 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504da90-fa1e-4132-bb1a-fcf610b1d2fb None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:53.338488 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504da90-fa1e-4132-bb1a-fcf610b1d2fb None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:53.339895 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 734/1473] 10.222.0.16 () {54 vars in 967 bytes} [Wed Aug 31 10:57:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:53.345047 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e81ad7-fb9d-47bc-8ccc-7eabf5ad9af7 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:53.345332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e81ad7-fb9d-47bc-8ccc-7eabf5ad9af7 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:53.345611 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e81ad7-fb9d-47bc-8ccc-7eabf5ad9af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:53.424281 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-66e81ad7-fb9d-47bc-8ccc-7eabf5ad9af7 None None] MFA Rules not processed for user `f2e72c8e7b444e978bf99c6404febcd2`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:53.425500 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-66e81ad7-fb9d-47bc-8ccc-7eabf5ad9af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:53.447183 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:53.507357 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 740/1474] 10.222.0.16 () {58 vars in 1064 bytes} [Wed Aug 31 10:57:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:53.515121 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f504da90-fa1e-4132-bb1a-fcf610b1d2fb None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:57:53.516689 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-f504da90-fa1e-4132-bb1a-fcf610b1d2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:53.623489 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f504da90-fa1e-4132-bb1a-fcf610b1d2fb None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:53.701023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb16ed8-38cd-4e99-bbf9-3fefc7681cdf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2e72c8e7b444e978bf99c6404febcd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:53.701629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb16ed8-38cd-4e99-bbf9-3fefc7681cdf service glance] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:53.701904 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb16ed8-38cd-4e99-bbf9-3fefc7681cdf service glance] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:53.702172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb16ed8-38cd-4e99-bbf9-3fefc7681cdf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:53.703801 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-fdb16ed8-38cd-4e99-bbf9-3fefc7681cdf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:53.757295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:53.813997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb16ed8-38cd-4e99-bbf9-3fefc7681cdf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:53.815478 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb16ed8-38cd-4e99-bbf9-3fefc7681cdf service glance] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:53.834561 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 735/1475] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:57:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:57:55.839134 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d1099-6a01-4886-afac-7224a6df4623 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:55.839690 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d1099-6a01-4886-afac-7224a6df4623 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:55.839690 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d1099-6a01-4886-afac-7224a6df4623 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:55.841000 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 741/1476] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:55.846167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef1d02-3af4-4c4c-9a0b-26fc61dd635d None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:55.846632 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef1d02-3af4-4c4c-9a0b-26fc61dd635d None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:55.846768 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef1d02-3af4-4c4c-9a0b-26fc61dd635d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:55.912194 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-16ef1d02-3af4-4c4c-9a0b-26fc61dd635d None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:55.913399 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-16ef1d02-3af4-4c4c-9a0b-26fc61dd635d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:56.015371 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 736/1477] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:57:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:56.284838 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d1099-6a01-4886-afac-7224a6df4623 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:57:56.286355 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-237d1099-6a01-4886-afac-7224a6df4623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:56.408965 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d1099-6a01-4886-afac-7224a6df4623 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:56.456769 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:56.500640 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d5713-ff15-40ab-a8f4-96863ae9f6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:56.501237 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d5713-ff15-40ab-a8f4-96863ae9f6b8 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:56.501514 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d5713-ff15-40ab-a8f4-96863ae9f6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:56.501775 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d5713-ff15-40ab-a8f4-96863ae9f6b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:56.502423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d5713-ff15-40ab-a8f4-96863ae9f6b8 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:56.504137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: WARNING py.warnings [None req-b36d5713-ff15-40ab-a8f4-96863ae9f6b8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 10:57:56.504137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: warnings.warn(msg) Aug 31 10:57:56.504137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: Aug 31 10:57:56.505273 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d5713-ff15-40ab-a8f4-96863ae9f6b8 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:56.524046 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 742/1478] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 31 10:57:56 2022] GET /identity/v3/projects => generated 2553 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:57:58.476746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871bdbce-91f0-427c-8c28-d31ab0765b02 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:58.477392 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871bdbce-91f0-427c-8c28-d31ab0765b02 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:58.477392 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871bdbce-91f0-427c-8c28-d31ab0765b02 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:58.479433 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 737/1479] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:57:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:58.484918 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c2ce4d-4952-48ed-9667-a1ddbf98ead0 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:58.485351 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c2ce4d-4952-48ed-9667-a1ddbf98ead0 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:58.485454 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c2ce4d-4952-48ed-9667-a1ddbf98ead0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:58.551356 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-a9c2ce4d-4952-48ed-9667-a1ddbf98ead0 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:58.552546 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a9c2ce4d-4952-48ed-9667-a1ddbf98ead0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:58.647946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 743/1480] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:58.941926 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871bdbce-91f0-427c-8c28-d31ab0765b02 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:57:58.943547 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-871bdbce-91f0-427c-8c28-d31ab0765b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:58.953967 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:59.091027 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871bdbce-91f0-427c-8c28-d31ab0765b02 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:59.174511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:59.207873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a66cf8-8f6c-4662-99e9-f4f02601612e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:59.208496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a66cf8-8f6c-4662-99e9-f4f02601612e service nova] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:59.209006 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a66cf8-8f6c-4662-99e9-f4f02601612e service nova] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:59.209322 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a66cf8-8f6c-4662-99e9-f4f02601612e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:59.211236 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-04a66cf8-8f6c-4662-99e9-f4f02601612e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:59.303669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a66cf8-8f6c-4662-99e9-f4f02601612e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:59.305214 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a66cf8-8f6c-4662-99e9-f4f02601612e service nova] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:59.323880 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 738/1481] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:57:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:58:01.430801 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529d17a-4a8c-46e6-aba7-21e319387a07 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:01.431902 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529d17a-4a8c-46e6-aba7-21e319387a07 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:01.431902 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529d17a-4a8c-46e6-aba7-21e319387a07 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:01.433202 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 744/1482] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:58:01.438572 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff75129-afaf-492d-9bf5-b799fb96c68a None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:01.438980 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff75129-afaf-492d-9bf5-b799fb96c68a None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:01.439121 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff75129-afaf-492d-9bf5-b799fb96c68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:01.504297 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-0ff75129-afaf-492d-9bf5-b799fb96c68a None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:58:01.505505 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-0ff75129-afaf-492d-9bf5-b799fb96c68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:01.550631 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:01.600172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 739/1483] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:58:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:58:01.887707 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529d17a-4a8c-46e6-aba7-21e319387a07 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:58:01.889254 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2529d17a-4a8c-46e6-aba7-21e319387a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:01.913348 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:01.995237 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529d17a-4a8c-46e6-aba7-21e319387a07 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:58:02.067447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e30385-386c-4e44-aa2e-885a472fb075 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:58:02.068056 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e30385-386c-4e44-aa2e-885a472fb075 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:02.068331 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e30385-386c-4e44-aa2e-885a472fb075 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:02.068599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e30385-386c-4e44-aa2e-885a472fb075 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:02.070203 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-79e30385-386c-4e44-aa2e-885a472fb075 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:02.144708 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:02.179682 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e30385-386c-4e44-aa2e-885a472fb075 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:58:02.181185 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e30385-386c-4e44-aa2e-885a472fb075 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:58:02.199220 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 745/1484] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:58:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:58:04.156331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73433976-cdab-41e9-a1dd-ea64e6a40a61 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:04.156841 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73433976-cdab-41e9-a1dd-ea64e6a40a61 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:04.156841 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73433976-cdab-41e9-a1dd-ea64e6a40a61 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:04.158145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 740/1485] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:58:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:58:04.163364 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99937a46-c4f3-47b9-a442-24807e3e6120 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:04.163806 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99937a46-c4f3-47b9-a442-24807e3e6120 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:04.163908 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99937a46-c4f3-47b9-a442-24807e3e6120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:04.233674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-99937a46-c4f3-47b9-a442-24807e3e6120 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:58:04.234914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-99937a46-c4f3-47b9-a442-24807e3e6120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:04.332978 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 746/1486] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:58:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:58:04.619466 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73433976-cdab-41e9-a1dd-ea64e6a40a61 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:58:04.620992 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-73433976-cdab-41e9-a1dd-ea64e6a40a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:04.723681 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73433976-cdab-41e9-a1dd-ea64e6a40a61 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:58:04.797544 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:04.799331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6266c1-e209-4364-affe-bc6811b40962 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:58:04.799931 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6266c1-e209-4364-affe-bc6811b40962 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:04.800203 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6266c1-e209-4364-affe-bc6811b40962 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:04.800490 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6266c1-e209-4364-affe-bc6811b40962 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:04.802098 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-bf6266c1-e209-4364-affe-bc6811b40962 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:04.909755 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6266c1-e209-4364-affe-bc6811b40962 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:58:04.911120 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6266c1-e209-4364-affe-bc6811b40962 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:58:04.929358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 741/1487] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:58:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:58:06.928992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb633a6-ae78-4d50-8a1d-21aff9976d05 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:06.929510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb633a6-ae78-4d50-8a1d-21aff9976d05 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:06.929623 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb633a6-ae78-4d50-8a1d-21aff9976d05 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:06.930992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 747/1488] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:58:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:58:06.936451 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c1f83-d27a-40df-84e1-67a1e5045dbf None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:06.936917 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c1f83-d27a-40df-84e1-67a1e5045dbf None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:06.937023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c1f83-d27a-40df-84e1-67a1e5045dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:07.004876 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-638c1f83-d27a-40df-84e1-67a1e5045dbf None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:58:07.006014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-638c1f83-d27a-40df-84e1-67a1e5045dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:07.063791 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:07.116341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 742/1489] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:58:07.411859 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb633a6-ae78-4d50-8a1d-21aff9976d05 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:58:07.413402 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-0bb633a6-ae78-4d50-8a1d-21aff9976d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:07.450323 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:07.517922 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb633a6-ae78-4d50-8a1d-21aff9976d05 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:58:07.591811 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b2794-6b2e-4ec2-b6d3-2f4336bf3f63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:58:07.592414 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b2794-6b2e-4ec2-b6d3-2f4336bf3f63 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:07.592688 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b2794-6b2e-4ec2-b6d3-2f4336bf3f63 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:07.592958 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b2794-6b2e-4ec2-b6d3-2f4336bf3f63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:07.594590 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-813b2794-6b2e-4ec2-b6d3-2f4336bf3f63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:07.676199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:07.706439 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b2794-6b2e-4ec2-b6d3-2f4336bf3f63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:58:07.708105 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b2794-6b2e-4ec2-b6d3-2f4336bf3f63 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:58:07.725748 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 748/1490] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:58:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:58:10.084776 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28303a8-4b14-4536-930f-b0e2bc1a8a03 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:10.085341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28303a8-4b14-4536-930f-b0e2bc1a8a03 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:10.085341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28303a8-4b14-4536-930f-b0e2bc1a8a03 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:10.086730 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 743/1491] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 31 10:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:58:10.092654 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc926e4-2080-4529-b02e-bd0c63b221da None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:10.093105 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc926e4-2080-4529-b02e-bd0c63b221da None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:10.093202 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc926e4-2080-4529-b02e-bd0c63b221da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:10.162705 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-cfc926e4-2080-4529-b02e-bd0c63b221da None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:58:10.163928 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-cfc926e4-2080-4529-b02e-bd0c63b221da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:10.261751 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 749/1492] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 31 10:58:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:58:10.555053 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28303a8-4b14-4536-930f-b0e2bc1a8a03 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:58:10.556776 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a28303a8-4b14-4536-930f-b0e2bc1a8a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:10.662645 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28303a8-4b14-4536-930f-b0e2bc1a8a03 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:58:10.736370 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d68a8a-a735-4ae6-97b0-8998c8b1e088 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:58:10.736961 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d68a8a-a735-4ae6-97b0-8998c8b1e088 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:10.737233 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d68a8a-a735-4ae6-97b0-8998c8b1e088 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:10.737567 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d68a8a-a735-4ae6-97b0-8998c8b1e088 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:10.739203 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-49d68a8a-a735-4ae6-97b0-8998c8b1e088 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:10.749728 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:10.850185 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d68a8a-a735-4ae6-97b0-8998c8b1e088 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:58:10.851630 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d68a8a-a735-4ae6-97b0-8998c8b1e088 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:58:10.870447 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 744/1493] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:58:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:58:20.260065 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=82 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:20.867895 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=50 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:36.621081 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5593852-e601-43f5-bb6d-3b4c3e5c75b2 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:36.621693 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5593852-e601-43f5-bb6d-3b4c3e5c75b2 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:36.621693 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5593852-e601-43f5-bb6d-3b4c3e5c75b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:36.698156 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-a5593852-e601-43f5-bb6d-3b4c3e5c75b2 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:59:36.699513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a5593852-e601-43f5-bb6d-3b4c3e5c75b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:36.802462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 750/1494] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 10:59:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:59:36.810526 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c903a10-7565-451b-b3b5-23604b3626d7 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:36.811060 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c903a10-7565-451b-b3b5-23604b3626d7 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:36.811060 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c903a10-7565-451b-b3b5-23604b3626d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:36.893371 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-4c903a10-7565-451b-b3b5-23604b3626d7 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:59:36.894532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-4c903a10-7565-451b-b3b5-23604b3626d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:36.995351 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 745/1495] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 10:59:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:59:37.000493 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5593852-e601-43f5-bb6d-3b4c3e5c75b2 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:59:37.001724 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a5593852-e601-43f5-bb6d-3b4c3e5c75b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:37.013755 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5593852-e601-43f5-bb6d-3b4c3e5c75b2 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:59:37.080009 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:37.116305 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6578d-a233-4788-b5bb-294ca5de7605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:59:37.116912 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6578d-a233-4788-b5bb-294ca5de7605 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:37.120092 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6578d-a233-4788-b5bb-294ca5de7605 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:37.120092 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6578d-a233-4788-b5bb-294ca5de7605 admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:37.120092 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b6578d-a233-4788-b5bb-294ca5de7605 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:59:37.120092 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b6578d-a233-4788-b5bb-294ca5de7605 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:59:37.120092 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: WARNING py.warnings [None req-74b6578d-a233-4788-b5bb-294ca5de7605 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 10:59:37.120092 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: warnings.warn(msg) Aug 31 10:59:37.120092 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: Aug 31 10:59:37.120624 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b6578d-a233-4788-b5bb-294ca5de7605 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:59:37.132219 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 751/1496] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 10:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:59:37.141252 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c903a10-7565-451b-b3b5-23604b3626d7 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:59:37.152325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c903a10-7565-451b-b3b5-23604b3626d7 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:59:37.214834 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:37.249542 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f225ed-4441-4027-8df4-46dc22de5586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:59:37.250501 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f225ed-4441-4027-8df4-46dc22de5586 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:37.251030 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f225ed-4441-4027-8df4-46dc22de5586 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:37.251381 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f225ed-4441-4027-8df4-46dc22de5586 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:37.252086 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f225ed-4441-4027-8df4-46dc22de5586 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:59:37.254170 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: WARNING py.warnings [None req-83f225ed-4441-4027-8df4-46dc22de5586 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 10:59:37.254170 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: warnings.warn(msg) Aug 31 10:59:37.254170 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: Aug 31 10:59:37.255485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f225ed-4441-4027-8df4-46dc22de5586 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:59:37.387965 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 746/1497] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 10:59:37 2022] POST /identity/v3/projects => generated 379 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:59:37.392320 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6578d-a233-4788-b5bb-294ca5de7605 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:59:37.394168 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-74b6578d-a233-4788-b5bb-294ca5de7605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:37.429845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6578d-a233-4788-b5bb-294ca5de7605 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:59:37.528926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141a5d5-3d6a-4eff-bbe5-483c9d807100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:59:37.529540 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141a5d5-3d6a-4eff-bbe5-483c9d807100 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:37.529833 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141a5d5-3d6a-4eff-bbe5-483c9d807100 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:37.530100 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141a5d5-3d6a-4eff-bbe5-483c9d807100 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:37.530773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0141a5d5-3d6a-4eff-bbe5-483c9d807100 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:59:37.531782 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: WARNING py.warnings [None req-0141a5d5-3d6a-4eff-bbe5-483c9d807100 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 10:59:37.531782 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: warnings.warn(msg) Aug 31 10:59:37.531782 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: Aug 31 10:59:37.533035 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0141a5d5-3d6a-4eff-bbe5-483c9d807100 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:59:37.683738 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 752/1498] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 10:59:37 2022] POST /identity/v3/users => generated 290 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:59:37.688077 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f225ed-4441-4027-8df4-46dc22de5586 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:59:37.690613 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-83f225ed-4441-4027-8df4-46dc22de5586 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:37.758414 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f225ed-4441-4027-8df4-46dc22de5586 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:59:37.846410 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:37.857388 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893de59e-70d3-495c-847e-0ebc2b025e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:59:37.861010 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893de59e-70d3-495c-847e-0ebc2b025e02 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:37.861010 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893de59e-70d3-495c-847e-0ebc2b025e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:37.861010 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893de59e-70d3-495c-847e-0ebc2b025e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:37.861010 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893de59e-70d3-495c-847e-0ebc2b025e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:59:37.861010 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: WARNING py.warnings [None req-893de59e-70d3-495c-847e-0ebc2b025e02 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 10:59:37.861010 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: warnings.warn(msg) Aug 31 10:59:37.861010 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: Aug 31 10:59:37.861614 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893de59e-70d3-495c-847e-0ebc2b025e02 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:59:37.874353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 747/1499] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 10:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:59:37.878546 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141a5d5-3d6a-4eff-bbe5-483c9d807100 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:59:37.880128 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-0141a5d5-3d6a-4eff-bbe5-483c9d807100 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:37.892842 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141a5d5-3d6a-4eff-bbe5-483c9d807100 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:59:37.918819 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:37.993376 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e83ab-3fde-4ad3-bc73-26743bef0ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:59:37.993900 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e83ab-3fde-4ad3-bc73-26743bef0ef5 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:37.994175 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e83ab-3fde-4ad3-bc73-26743bef0ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:37.994475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e83ab-3fde-4ad3-bc73-26743bef0ef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:37.995918 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e83ab-3fde-4ad3-bc73-26743bef0ef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:59:37.997617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: WARNING py.warnings [None req-d75e83ab-3fde-4ad3-bc73-26743bef0ef5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 10:59:37.997617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: warnings.warn(msg) Aug 31 10:59:37.997617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: Aug 31 10:59:37.999026 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e83ab-3fde-4ad3-bc73-26743bef0ef5 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:59:38.013370 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 753/1500] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 10:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:59:38.020097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893de59e-70d3-495c-847e-0ebc2b025e02 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:59:38.021616 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-893de59e-70d3-495c-847e-0ebc2b025e02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:38.055407 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893de59e-70d3-495c-847e-0ebc2b025e02 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:59:38.159450 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51b735-acfe-4c4c-a46e-0338be0d8e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:59:38.160108 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51b735-acfe-4c4c-a46e-0338be0d8e5b admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:38.160382 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51b735-acfe-4c4c-a46e-0338be0d8e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:38.160649 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51b735-acfe-4c4c-a46e-0338be0d8e5b admin admin] PATH_INFO: `/v3/projects/add4f84c63fe444eb2393f4134b2ae50/users/0a09ea32564a43288a008d27bf03dd64/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:38.183607 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb51b735-acfe-4c4c-a46e-0338be0d8e5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=add4f84c63fe444eb2393f4134b2ae50, user_id=0a09ea32564a43288a008d27bf03dd64, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:59:38.184637 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: WARNING py.warnings [None req-fb51b735-acfe-4c4c-a46e-0338be0d8e5b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 10:59:38.184637 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: warnings.warn(msg) Aug 31 10:59:38.184637 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: Aug 31 10:59:38.186607 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb51b735-acfe-4c4c-a46e-0338be0d8e5b admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:59:38.265750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 748/1501] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 10:59:38 2022] PUT /identity/v3/projects/add4f84c63fe444eb2393f4134b2ae50/users/0a09ea32564a43288a008d27bf03dd64/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:59:38.278786 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7bdcca-1f24-4157-96ec-2c47db38611f None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:38.279071 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7bdcca-1f24-4157-96ec-2c47db38611f None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:38.279421 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7bdcca-1f24-4157-96ec-2c47db38611f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:38.337985 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-ef7bdcca-1f24-4157-96ec-2c47db38611f None None] MFA Rules not processed for user `0a09ea32564a43288a008d27bf03dd64`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:59:38.339300 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-ef7bdcca-1f24-4157-96ec-2c47db38611f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:38.418955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 754/1502] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 10:59:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:59:38.431957 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3775f-1611-4b16-adc6-bafab155c009 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:38.432242 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3775f-1611-4b16-adc6-bafab155c009 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:38.433237 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3775f-1611-4b16-adc6-bafab155c009 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:38.435087 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 749/1503] 10.222.0.16 () {54 vars in 964 bytes} [Wed Aug 31 10:59:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:59:38.444221 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26dfbcd-63b0-4e25-b92b-5db1b7fd0653 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:38.444514 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26dfbcd-63b0-4e25-b92b-5db1b7fd0653 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:38.444784 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26dfbcd-63b0-4e25-b92b-5db1b7fd0653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:38.478635 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:38.538352 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-c26dfbcd-63b0-4e25-b92b-5db1b7fd0653 None None] MFA Rules not processed for user `49f9aeb9857b45d68c3baa8fc4acc5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:59:38.539521 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-c26dfbcd-63b0-4e25-b92b-5db1b7fd0653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:38.617311 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 755/1504] 10.222.0.16 () {58 vars in 1061 bytes} [Wed Aug 31 10:59:38 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:59:38.627576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3775f-1611-4b16-adc6-bafab155c009 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:59:38.629145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-6df3775f-1611-4b16-adc6-bafab155c009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:38.680401 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:38.737362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3775f-1611-4b16-adc6-bafab155c009 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:59:38.812251 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a409d-a34d-44f5-9ae1-3e335a3ec33b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:59:38.812907 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a409d-a34d-44f5-9ae1-3e335a3ec33b service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:38.813189 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a409d-a34d-44f5-9ae1-3e335a3ec33b service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:38.813459 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a409d-a34d-44f5-9ae1-3e335a3ec33b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:38.815148 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-551a409d-a34d-44f5-9ae1-3e335a3ec33b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:38.918704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a409d-a34d-44f5-9ae1-3e335a3ec33b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:59:38.920174 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a409d-a34d-44f5-9ae1-3e335a3ec33b service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:59:38.923825 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:38.939287 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 750/1505] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:59:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:59:38.987127 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4be4b1-172f-4c58-8b3a-0632173e4885 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:38.987596 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4be4b1-172f-4c58-8b3a-0632173e4885 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:38.987702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4be4b1-172f-4c58-8b3a-0632173e4885 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:38.988951 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 756/1506] 10.222.0.16 () {52 vars in 830 bytes} [Wed Aug 31 10:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:59:38.992763 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a409d-a34d-44f5-9ae1-3e335a3ec33b service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:59:38.994035 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-551a409d-a34d-44f5-9ae1-3e335a3ec33b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:39.101713 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a409d-a34d-44f5-9ae1-3e335a3ec33b service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:59:39.199334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a746168d-f838-4d1e-92b3-a789f1080799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:59:39.199966 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746168d-f838-4d1e-92b3-a789f1080799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:39.200239 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746168d-f838-4d1e-92b3-a789f1080799 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:39.200504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746168d-f838-4d1e-92b3-a789f1080799 admin admin] PATH_INFO: `/v3/users/0a09ea32564a43288a008d27bf03dd64` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:39.212723 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a746168d-f838-4d1e-92b3-a789f1080799 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a09ea32564a43288a008d27bf03dd64)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:59:39.213814 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: WARNING py.warnings [None req-a746168d-f838-4d1e-92b3-a789f1080799 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 10:59:39.213814 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: warnings.warn(msg) Aug 31 10:59:39.213814 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: Aug 31 10:59:39.215225 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a746168d-f838-4d1e-92b3-a789f1080799 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:59:39.224979 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:39.429782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-a746168d-f838-4d1e-92b3-a789f1080799 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a09ea32564a43288a008d27bf03dd64'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:59:39.504725 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-a746168d-f838-4d1e-92b3-a789f1080799 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a09ea32564a43288a008d27bf03dd64'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:59:39.505255 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-a746168d-f838-4d1e-92b3-a789f1080799 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a09ea32564a43288a008d27bf03dd64'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:59:39.505677 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-a746168d-f838-4d1e-92b3-a789f1080799 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a09ea32564a43288a008d27bf03dd64'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:59:39.529640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-a746168d-f838-4d1e-92b3-a789f1080799 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a09ea32564a43288a008d27bf03dd64'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:59:39.549582 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 751/1507] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 10:59:38 2022] DELETE /identity/v3/users/0a09ea32564a43288a008d27bf03dd64 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:59:39.554243 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea3b18-43a7-47d2-b7f4-1524cc7754c2 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:39.554561 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea3b18-43a7-47d2-b7f4-1524cc7754c2 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:39.554861 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea3b18-43a7-47d2-b7f4-1524cc7754c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:39.617440 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-a2ea3b18-43a7-47d2-b7f4-1524cc7754c2 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:59:39.618683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a2ea3b18-43a7-47d2-b7f4-1524cc7754c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:39.698022 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:39.711444 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 757/1508] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 10:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:59:39.722466 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a746168d-f838-4d1e-92b3-a789f1080799 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:59:39.723713 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a746168d-f838-4d1e-92b3-a789f1080799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:39.825576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a746168d-f838-4d1e-92b3-a789f1080799 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:59:39.884435 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:39.903953 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856b573-3194-45d9-8869-b54e424e81c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:59:39.904570 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856b573-3194-45d9-8869-b54e424e81c6 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:39.904847 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856b573-3194-45d9-8869-b54e424e81c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:39.905116 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856b573-3194-45d9-8869-b54e424e81c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:39.906753 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-1856b573-3194-45d9-8869-b54e424e81c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:40.028835 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1856b573-3194-45d9-8869-b54e424e81c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:59:40.030288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1856b573-3194-45d9-8869-b54e424e81c6 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:59:40.052098 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 752/1509] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 10:59:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:59:40.572831 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea3b18-43a7-47d2-b7f4-1524cc7754c2 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:59:40.575198 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a2ea3b18-43a7-47d2-b7f4-1524cc7754c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:40.587062 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea3b18-43a7-47d2-b7f4-1524cc7754c2 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:59:40.693350 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21741aee-952f-4a2d-aea2-7c2022da7061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:59:40.694024 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21741aee-952f-4a2d-aea2-7c2022da7061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:40.694302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21741aee-952f-4a2d-aea2-7c2022da7061 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:40.694601 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21741aee-952f-4a2d-aea2-7c2022da7061 admin admin] PATH_INFO: `/v3/projects/add4f84c63fe444eb2393f4134b2ae50` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:40.706128 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21741aee-952f-4a2d-aea2-7c2022da7061 admin admin] RBAC: Authorizing `identity:delete_project(project_id=add4f84c63fe444eb2393f4134b2ae50)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:59:40.707195 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: WARNING py.warnings [None req-21741aee-952f-4a2d-aea2-7c2022da7061 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 10:59:40.707195 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: warnings.warn(msg) Aug 31 10:59:40.707195 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: Aug 31 10:59:40.708406 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21741aee-952f-4a2d-aea2-7c2022da7061 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:59:40.854304 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-21741aee-952f-4a2d-aea2-7c2022da7061 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'add4f84c63fe444eb2393f4134b2ae50', 'request_id': 'req-21741aee-952f-4a2d-aea2-7c2022da7061'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:59:40.959396 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-21741aee-952f-4a2d-aea2-7c2022da7061 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'add4f84c63fe444eb2393f4134b2ae50', 'request_id': 'req-21741aee-952f-4a2d-aea2-7c2022da7061'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:59:40.967042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-21741aee-952f-4a2d-aea2-7c2022da7061 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'add4f84c63fe444eb2393f4134b2ae50', 'request_id': 'req-21741aee-952f-4a2d-aea2-7c2022da7061'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:59:41.000432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-21741aee-952f-4a2d-aea2-7c2022da7061 admin admin] The token cache is being invalidate because project add4f84c63fe444eb2393f4134b2ae50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:59:41.000718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-21741aee-952f-4a2d-aea2-7c2022da7061 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:59:41.001253 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-21741aee-952f-4a2d-aea2-7c2022da7061 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:59:41.002610 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 758/1510] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 10:59:40 2022] DELETE /identity/v3/projects/add4f84c63fe444eb2393f4134b2ae50 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:59:50.049836 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=62 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:50.998291 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:01:15.232819 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09ccb1-12ea-4f99-ba30-8012ca739faa None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:15.233439 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09ccb1-12ea-4f99-ba30-8012ca739faa None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:15.233439 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09ccb1-12ea-4f99-ba30-8012ca739faa None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:15.234804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 11:01:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:01:15.240341 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c49ea9-03d3-4ade-8e92-1aedcd914584 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:15.240822 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c49ea9-03d3-4ade-8e92-1aedcd914584 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:15.240925 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c49ea9-03d3-4ade-8e92-1aedcd914584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:15.326037 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-37c49ea9-03d3-4ade-8e92-1aedcd914584 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:01:15.327370 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-37c49ea9-03d3-4ade-8e92-1aedcd914584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:15.431202 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 11:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:01:15.588744 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09ccb1-12ea-4f99-ba30-8012ca739faa None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:01:15.593363 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-9e09ccb1-12ea-4f99-ba30-8012ca739faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:15.709973 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09ccb1-12ea-4f99-ba30-8012ca739faa None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:01:15.715483 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845274c-f7fb-436e-b87d-8d4c81161e6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:01:15.716150 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845274c-f7fb-436e-b87d-8d4c81161e6e service nova] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:15.716427 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845274c-f7fb-436e-b87d-8d4c81161e6e service nova] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:15.716700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845274c-f7fb-436e-b87d-8d4c81161e6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:15.725043 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d845274c-f7fb-436e-b87d-8d4c81161e6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:01:15.726696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d845274c-f7fb-436e-b87d-8d4c81161e6e service nova] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:01:15.737055 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 754/1513] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:01:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:01:16.570007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f6e42-64d7-402e-81d2-3af892099f71 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:16.570605 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f6e42-64d7-402e-81d2-3af892099f71 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:16.570605 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f6e42-64d7-402e-81d2-3af892099f71 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:16.571898 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 11:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:01:16.577388 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a5f00-4df9-41bf-992e-34120a88442c None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:16.577848 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a5f00-4df9-41bf-992e-34120a88442c None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:16.577964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a5f00-4df9-41bf-992e-34120a88442c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:16.598034 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-aa6a5f00-4df9-41bf-992e-34120a88442c None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:01:16.599277 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-aa6a5f00-4df9-41bf-992e-34120a88442c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:16.608010 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 11:01:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:01:16.768155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f6e42-64d7-402e-81d2-3af892099f71 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:01:16.782200 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f6e42-64d7-402e-81d2-3af892099f71 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:01:16.787600 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faea206-cf8a-4dd3-8064-1b3d2dc1945e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:01:16.788220 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faea206-cf8a-4dd3-8064-1b3d2dc1945e service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:16.788501 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faea206-cf8a-4dd3-8064-1b3d2dc1945e service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:16.788764 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faea206-cf8a-4dd3-8064-1b3d2dc1945e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:16.796801 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faea206-cf8a-4dd3-8064-1b3d2dc1945e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:01:16.798413 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faea206-cf8a-4dd3-8064-1b3d2dc1945e service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:01:16.807581 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 761/1516] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:01:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:01:17.552697 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd2525-7498-44ce-a919-b795ef760538 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:17.552977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd2525-7498-44ce-a919-b795ef760538 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:17.553251 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd2525-7498-44ce-a919-b795ef760538 None None] PATH_INFO: `/v3` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:17.554603 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 756/1517] 10.222.0.69 () {54 vars in 925 bytes} [Wed Aug 31 11:01:17 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:01:17.562635 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f334f64-32a0-40b8-a427-01c08ba4b748 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:17.562943 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f334f64-32a0-40b8-a427-01c08ba4b748 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:17.563216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f334f64-32a0-40b8-a427-01c08ba4b748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:17.622731 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-7f334f64-32a0-40b8-a427-01c08ba4b748 None None] MFA Rules not processed for user `f3db46a4b4e74cc0bb2a7bd6f0e99105`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:01:17.623923 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-7f334f64-32a0-40b8-a427-01c08ba4b748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:17.640307 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:01:17.702284 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 762/1518] 10.222.0.69 () {58 vars in 1015 bytes} [Wed Aug 31 11:01:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:01:26.593121 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=56 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:01:27.615861 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa4489-c835-43d2-9d10-c3ffccece7ff None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:27.616527 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa4489-c835-43d2-9d10-c3ffccece7ff None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:27.616527 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa4489-c835-43d2-9d10-c3ffccece7ff None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:27.617697 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 757/1519] 10.222.0.16 () {54 vars in 961 bytes} [Wed Aug 31 11:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:01:27.623782 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389b68b-e721-4997-adbb-6856efc9e318 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:27.624353 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389b68b-e721-4997-adbb-6856efc9e318 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:27.624353 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389b68b-e721-4997-adbb-6856efc9e318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:27.646632 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-7389b68b-e721-4997-adbb-6856efc9e318 None None] MFA Rules not processed for user `f3db46a4b4e74cc0bb2a7bd6f0e99105`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:01:27.647980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-7389b68b-e721-4997-adbb-6856efc9e318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:27.657603 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 763/1520] 10.222.0.16 () {58 vars in 1058 bytes} [Wed Aug 31 11:01:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:01:27.667150 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa4489-c835-43d2-9d10-c3ffccece7ff None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:01:27.682066 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa4489-c835-43d2-9d10-c3ffccece7ff None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:01:27.688779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42925cab-37ce-49da-ba82-bd30914ef758 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3db46a4b4e74cc0bb2a7bd6f0e99105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:01:27.689296 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42925cab-37ce-49da-ba82-bd30914ef758 service placement] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:27.689579 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42925cab-37ce-49da-ba82-bd30914ef758 service placement] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:27.689843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42925cab-37ce-49da-ba82-bd30914ef758 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:27.701349 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42925cab-37ce-49da-ba82-bd30914ef758 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:01:27.703421 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42925cab-37ce-49da-ba82-bd30914ef758 service placement] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:01:27.714006 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 758/1521] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:01:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:01:28.086898 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc6d7d-185e-48bf-b81d-d5b64838e2ee None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:28.087216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc6d7d-185e-48bf-b81d-d5b64838e2ee None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:28.087493 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc6d7d-185e-48bf-b81d-d5b64838e2ee None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:28.088877 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 11:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:01:28.094334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed632b91-8b30-4e65-9181-aa3ce329a4a9 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:28.094657 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed632b91-8b30-4e65-9181-aa3ce329a4a9 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:28.094927 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed632b91-8b30-4e65-9181-aa3ce329a4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:28.114788 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-ed632b91-8b30-4e65-9181-aa3ce329a4a9 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:01:28.116964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-ed632b91-8b30-4e65-9181-aa3ce329a4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:28.124467 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 11:01:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:01:28.282898 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc6d7d-185e-48bf-b81d-d5b64838e2ee None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:01:28.298002 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc6d7d-185e-48bf-b81d-d5b64838e2ee None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:01:28.303663 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543a5c9-3b81-41a0-b52f-7c860269145b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:01:28.304269 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543a5c9-3b81-41a0-b52f-7c860269145b service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:28.304553 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543a5c9-3b81-41a0-b52f-7c860269145b service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:28.304824 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543a5c9-3b81-41a0-b52f-7c860269145b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:28.314754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8543a5c9-3b81-41a0-b52f-7c860269145b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:01:28.316362 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8543a5c9-3b81-41a0-b52f-7c860269145b service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:01:28.326782 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 765/1524] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:01:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:01:29.168865 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2189a46-4905-40c6-9b75-d795a006f99e None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:29.169383 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2189a46-4905-40c6-9b75-d795a006f99e None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:29.169383 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2189a46-4905-40c6-9b75-d795a006f99e None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:29.170707 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 11:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:01:29.175494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f885e-365f-43bd-bdeb-d748e45f339c None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:29.175884 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f885e-365f-43bd-bdeb-d748e45f339c None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:29.176037 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f885e-365f-43bd-bdeb-d748e45f339c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:29.194845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-391f885e-365f-43bd-bdeb-d748e45f339c None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:01:29.196016 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-391f885e-365f-43bd-bdeb-d748e45f339c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:29.205223 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 11:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:01:29.358730 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2189a46-4905-40c6-9b75-d795a006f99e None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:01:29.373337 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2189a46-4905-40c6-9b75-d795a006f99e None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:01:29.378739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af66bf-0701-43a5-b789-c4c1c8258646 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:01:29.379325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af66bf-0701-43a5-b789-c4c1c8258646 service nova] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:29.379600 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af66bf-0701-43a5-b789-c4c1c8258646 service nova] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:29.379870 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af66bf-0701-43a5-b789-c4c1c8258646 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:29.388725 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af66bf-0701-43a5-b789-c4c1c8258646 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:01:29.390379 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af66bf-0701-43a5-b789-c4c1c8258646 service nova] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:01:29.399913 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 761/1527] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:01:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:01:33.540434 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abee008-2ecb-4aa2-8315-8296fbb8ddb6 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:33.541088 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abee008-2ecb-4aa2-8315-8296fbb8ddb6 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:33.541088 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abee008-2ecb-4aa2-8315-8296fbb8ddb6 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:33.542302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 11:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:01:33.547958 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb822b1-b26d-450c-8f69-6f640af6f6c9 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:33.548431 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb822b1-b26d-450c-8f69-6f640af6f6c9 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:33.548538 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb822b1-b26d-450c-8f69-6f640af6f6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:33.568026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-ecb822b1-b26d-450c-8f69-6f640af6f6c9 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:01:33.569211 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-ecb822b1-b26d-450c-8f69-6f640af6f6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:33.577156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 11:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:01:33.875838 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abee008-2ecb-4aa2-8315-8296fbb8ddb6 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:01:33.877746 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-0abee008-2ecb-4aa2-8315-8296fbb8ddb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:33.961383 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abee008-2ecb-4aa2-8315-8296fbb8ddb6 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:01:33.965542 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd5f11-a362-45c2-89df-2311a39f4690 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:01:33.966199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd5f11-a362-45c2-89df-2311a39f4690 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:33.966506 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd5f11-a362-45c2-89df-2311a39f4690 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:33.966777 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd5f11-a362-45c2-89df-2311a39f4690 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:33.975742 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecd5f11-a362-45c2-89df-2311a39f4690 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:01:33.977169 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecd5f11-a362-45c2-89df-2311a39f4690 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:01:33.987673 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 768/1530] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:01:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:01:35.935340 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cfbeb-db78-4225-8e29-e13e8db28ecf None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:35.935919 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cfbeb-db78-4225-8e29-e13e8db28ecf None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:35.935919 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cfbeb-db78-4225-8e29-e13e8db28ecf None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:35.937230 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 11:01:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:01:35.942604 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3058f4-7a60-43af-bb36-c1fa103edfd4 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:35.943040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3058f4-7a60-43af-bb36-c1fa103edfd4 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:35.943146 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3058f4-7a60-43af-bb36-c1fa103edfd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:35.962976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-9e3058f4-7a60-43af-bb36-c1fa103edfd4 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:01:35.964178 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-9e3058f4-7a60-43af-bb36-c1fa103edfd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:35.972243 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 11:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:01:36.263163 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136cfbeb-db78-4225-8e29-e13e8db28ecf None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:01:36.273910 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136cfbeb-db78-4225-8e29-e13e8db28ecf None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:01:36.280960 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aaa617-3345-40c1-9992-b2efca3f1640 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:01:36.280960 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aaa617-3345-40c1-9992-b2efca3f1640 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:36.280960 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aaa617-3345-40c1-9992-b2efca3f1640 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:36.280960 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aaa617-3345-40c1-9992-b2efca3f1640 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:36.288012 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aaa617-3345-40c1-9992-b2efca3f1640 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:01:36.289422 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aaa617-3345-40c1-9992-b2efca3f1640 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:01:36.298918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 764/1533] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:01:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:01:39.832086 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdeee46-1acf-4fb8-a1ba-54fecf91d067 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:39.832643 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdeee46-1acf-4fb8-a1ba-54fecf91d067 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:39.832643 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdeee46-1acf-4fb8-a1ba-54fecf91d067 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:39.833997 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 11:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:01:39.839331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c119f-4323-4143-8849-ed63b066bd12 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:39.840874 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c119f-4323-4143-8849-ed63b066bd12 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:39.840874 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c119f-4323-4143-8849-ed63b066bd12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:39.860923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-c02c119f-4323-4143-8849-ed63b066bd12 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:01:39.862241 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-c02c119f-4323-4143-8849-ed63b066bd12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:39.870357 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 11:01:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:01:40.430626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdeee46-1acf-4fb8-a1ba-54fecf91d067 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:01:40.432299 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2cdeee46-1acf-4fb8-a1ba-54fecf91d067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:40.502282 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:01:40.516106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdeee46-1acf-4fb8-a1ba-54fecf91d067 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:01:40.519696 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4152199-48b4-4754-9579-a57b053a84c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:01:40.520301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4152199-48b4-4754-9579-a57b053a84c0 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:40.520579 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4152199-48b4-4754-9579-a57b053a84c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:40.520851 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4152199-48b4-4754-9579-a57b053a84c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:40.531067 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4152199-48b4-4754-9579-a57b053a84c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:01:40.532505 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4152199-48b4-4754-9579-a57b053a84c0 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:01:40.543741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 771/1536] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:01:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:01:42.485094 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11622901-1229-4bba-946d-68aa919d0185 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:42.485673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11622901-1229-4bba-946d-68aa919d0185 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:42.485673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11622901-1229-4bba-946d-68aa919d0185 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:42.487010 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 11:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:01:42.492182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61c2f5-dc3d-4fb3-9d42-abf079e4b517 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:42.492588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61c2f5-dc3d-4fb3-9d42-abf079e4b517 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:42.492742 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61c2f5-dc3d-4fb3-9d42-abf079e4b517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:42.516327 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-ba61c2f5-dc3d-4fb3-9d42-abf079e4b517 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:01:42.517519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-ba61c2f5-dc3d-4fb3-9d42-abf079e4b517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:42.525330 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 11:01:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:01:43.100423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11622901-1229-4bba-946d-68aa919d0185 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:01:43.102239 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-11622901-1229-4bba-946d-68aa919d0185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:43.116642 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11622901-1229-4bba-946d-68aa919d0185 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:01:43.120243 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad109b-0d54-499c-931d-8075e1950148 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:01:43.120904 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad109b-0d54-499c-931d-8075e1950148 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:43.121347 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad109b-0d54-499c-931d-8075e1950148 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:43.121622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad109b-0d54-499c-931d-8075e1950148 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:43.135704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad109b-0d54-499c-931d-8075e1950148 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:01:43.138283 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad109b-0d54-499c-931d-8075e1950148 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:01:43.150271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 767/1539] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:01:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:01:52.511117 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=11 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:01:53.146144 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=33 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:11.693801 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19604c8-4db7-4b82-9a92-ba7e103d033c None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:11.694922 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19604c8-4db7-4b82-9a92-ba7e103d033c None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:11.695425 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19604c8-4db7-4b82-9a92-ba7e103d033c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:11.719188 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443985b6-423e-46c2-bd54-613844892f4d None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:11.720217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443985b6-423e-46c2-bd54-613844892f4d None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:11.720721 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443985b6-423e-46c2-bd54-613844892f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:11.724856 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-f19604c8-4db7-4b82-9a92-ba7e103d033c None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:11.726212 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-f19604c8-4db7-4b82-9a92-ba7e103d033c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:11.744182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 773/1540] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:11.755061 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e49af3-47a7-4633-88e5-33b3dfac3154 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:11.755837 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e49af3-47a7-4633-88e5-33b3dfac3154 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:11.756588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e49af3-47a7-4633-88e5-33b3dfac3154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:11.759423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-443985b6-423e-46c2-bd54-613844892f4d None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:11.760644 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-443985b6-423e-46c2-bd54-613844892f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:11.780656 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 768/1541] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:03:11.787511 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-20e49af3-47a7-4633-88e5-33b3dfac3154 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:11.788906 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad742e2-657e-47b1-bb45-73cd21332c1e None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:11.789191 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad742e2-657e-47b1-bb45-73cd21332c1e None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:11.789465 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad742e2-657e-47b1-bb45-73cd21332c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:11.789729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-20e49af3-47a7-4633-88e5-33b3dfac3154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:11.803739 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 774/1542] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:03:11.808967 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e49af3-47a7-4633-88e5-33b3dfac3154 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:11.824789 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e49af3-47a7-4633-88e5-33b3dfac3154 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:11.830336 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c3e8d-51e7-4691-b3c2-3c5a90677780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:11.831391 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c3e8d-51e7-4691-b3c2-3c5a90677780 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:11.831695 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-2ad742e2-657e-47b1-bb45-73cd21332c1e None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:11.832484 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c3e8d-51e7-4691-b3c2-3c5a90677780 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:11.832939 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-2ad742e2-657e-47b1-bb45-73cd21332c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:11.833328 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c3e8d-51e7-4691-b3c2-3c5a90677780 admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:11.834308 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c3e8d-51e7-4691-b3c2-3c5a90677780 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:03:11.834873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c3e8d-51e7-4691-b3c2-3c5a90677780 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:11.836755 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c3e8d-51e7-4691-b3c2-3c5a90677780 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:11.857827 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 769/1543] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:03:11.858169 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 775/1544] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:03:11.867626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad742e2-657e-47b1-bb45-73cd21332c1e None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:11.870652 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c3e8d-51e7-4691-b3c2-3c5a90677780 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:11.877660 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad742e2-657e-47b1-bb45-73cd21332c1e None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:11.889322 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c3e8d-51e7-4691-b3c2-3c5a90677780 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:11.893851 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d34ca-8613-4742-bbea-59f2f4cfecf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:11.894512 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d34ca-8613-4742-bbea-59f2f4cfecf4 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:11.894787 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d34ca-8613-4742-bbea-59f2f4cfecf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:11.895055 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d34ca-8613-4742-bbea-59f2f4cfecf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:11.895729 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d34ca-8613-4742-bbea-59f2f4cfecf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:03:11.896024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d34ca-8613-4742-bbea-59f2f4cfecf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:11.897315 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: WARNING py.warnings [None req-501d34ca-8613-4742-bbea-59f2f4cfecf4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:03:11.897315 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: warnings.warn(msg) Aug 31 11:03:11.897315 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: Aug 31 11:03:11.897607 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0366a9-3624-4edb-bd3e-7cda574f831a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:11.898256 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0366a9-3624-4edb-bd3e-7cda574f831a admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:11.898773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0366a9-3624-4edb-bd3e-7cda574f831a admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:11.899230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0366a9-3624-4edb-bd3e-7cda574f831a admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:11.900067 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0366a9-3624-4edb-bd3e-7cda574f831a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:11.901294 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: WARNING py.warnings [None req-7d0366a9-3624-4edb-bd3e-7cda574f831a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:03:11.901294 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: warnings.warn(msg) Aug 31 11:03:11.901294 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: Aug 31 11:03:11.903306 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d34ca-8613-4742-bbea-59f2f4cfecf4 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:11.908812 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0366a9-3624-4edb-bd3e-7cda574f831a admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:11.922958 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 770/1545] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:03:11.924875 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6651865-f55c-4842-af5b-9c8b96621fbf None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:11.925365 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6651865-f55c-4842-af5b-9c8b96621fbf None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:11.925823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6651865-f55c-4842-af5b-9c8b96621fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:11.966479 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-a6651865-f55c-4842-af5b-9c8b96621fbf None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:11.966479 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a6651865-f55c-4842-af5b-9c8b96621fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:11.981844 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 771/1546] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:11.983417 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6651865-f55c-4842-af5b-9c8b96621fbf None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:12.002518 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6651865-f55c-4842-af5b-9c8b96621fbf None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:12.010653 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953cc1c-3638-4abe-963f-8f8e37cb7f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:12.011725 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953cc1c-3638-4abe-963f-8f8e37cb7f85 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:12.012211 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953cc1c-3638-4abe-963f-8f8e37cb7f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:12.012682 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953cc1c-3638-4abe-963f-8f8e37cb7f85 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:12.013536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953cc1c-3638-4abe-963f-8f8e37cb7f85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:12.015669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953cc1c-3638-4abe-963f-8f8e37cb7f85 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:12.057644 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 776/1547] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:03:11 2022] POST /identity/v3/projects => generated 383 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:03:12.059248 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb24c95-6ae5-4115-b3b2-75735cb39f42 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:12.059596 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb24c95-6ae5-4115-b3b2-75735cb39f42 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:12.060360 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb24c95-6ae5-4115-b3b2-75735cb39f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:12.090792 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-bdb24c95-6ae5-4115-b3b2-75735cb39f42 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:12.092003 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-bdb24c95-6ae5-4115-b3b2-75735cb39f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:12.170094 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 777/1548] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:12.170879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb24c95-6ae5-4115-b3b2-75735cb39f42 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:12.181626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb24c95-6ae5-4115-b3b2-75735cb39f42 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:12.189542 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3dc6ee-0e14-4644-9923-4adf3a6e090d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:12.191592 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3dc6ee-0e14-4644-9923-4adf3a6e090d admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:12.192550 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3dc6ee-0e14-4644-9923-4adf3a6e090d admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:12.192550 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3dc6ee-0e14-4644-9923-4adf3a6e090d admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:12.192807 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3dc6ee-0e14-4644-9923-4adf3a6e090d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:12.196294 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3dc6ee-0e14-4644-9923-4adf3a6e090d admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:12.201883 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 772/1549] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:03:11 2022] POST /identity/v3/projects => generated 375 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:12.206659 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408dade4-999e-40e0-be98-eda45798a161 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:12.207304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408dade4-999e-40e0-be98-eda45798a161 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:12.207587 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408dade4-999e-40e0-be98-eda45798a161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:12.232602 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-408dade4-999e-40e0-be98-eda45798a161 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:12.232602 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-408dade4-999e-40e0-be98-eda45798a161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:12.299909 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 773/1550] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:12.299909 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989f709-4325-432b-baa7-dc3a13fa4fa8 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:12.299909 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989f709-4325-432b-baa7-dc3a13fa4fa8 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:12.299909 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989f709-4325-432b-baa7-dc3a13fa4fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:12.306766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 778/1551] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:03:12 2022] POST /identity/v3/users => generated 292 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:12.307826 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3dc6ee-0e14-4644-9923-4adf3a6e090d admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:12.315069 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-2989f709-4325-432b-baa7-dc3a13fa4fa8 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:12.316276 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-2989f709-4325-432b-baa7-dc3a13fa4fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:12.318661 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3dc6ee-0e14-4644-9923-4adf3a6e090d admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:12.323305 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0c901-8ea6-4062-a6c0-8984a7711133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:12.325042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0c901-8ea6-4062-a6c0-8984a7711133 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:12.325175 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 774/1552] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:12.325259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0c901-8ea6-4062-a6c0-8984a7711133 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:12.325492 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0c901-8ea6-4062-a6c0-8984a7711133 admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:12.326207 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0c901-8ea6-4062-a6c0-8984a7711133 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:03:12.326349 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df62832-d79a-47e0-b165-3049c39b710b None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:12.326547 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0c901-8ea6-4062-a6c0-8984a7711133 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:12.326669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df62832-d79a-47e0-b165-3049c39b710b None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:12.326903 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df62832-d79a-47e0-b165-3049c39b710b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:12.328149 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0c901-8ea6-4062-a6c0-8984a7711133 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:12.345087 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 779/1553] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:03:12.346078 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0c901-8ea6-4062-a6c0-8984a7711133 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:12.353155 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-0df62832-d79a-47e0-b165-3049c39b710b None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:12.354719 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-0df62832-d79a-47e0-b165-3049c39b710b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:12.362668 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0c901-8ea6-4062-a6c0-8984a7711133 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:12.363620 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 775/1554] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:12.364866 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23f226-3a6f-4afe-a56d-e36037f84d1c None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:12.365182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23f226-3a6f-4afe-a56d-e36037f84d1c None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:12.365452 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23f226-3a6f-4afe-a56d-e36037f84d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:12.367571 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc6c8b1-1831-4707-a4e8-d0bcef0ae8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:12.368168 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6c8b1-1831-4707-a4e8-d0bcef0ae8e1 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:12.368441 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6c8b1-1831-4707-a4e8-d0bcef0ae8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:12.368706 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6c8b1-1831-4707-a4e8-d0bcef0ae8e1 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:12.369329 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc6c8b1-1831-4707-a4e8-d0bcef0ae8e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:12.371582 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc6c8b1-1831-4707-a4e8-d0bcef0ae8e1 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:12.385304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-1a23f226-3a6f-4afe-a56d-e36037f84d1c None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:12.387069 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-1a23f226-3a6f-4afe-a56d-e36037f84d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:12.395546 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 776/1555] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:12.400740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a246817-ee1c-404b-81c4-caf6a6a62581 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:12.401064 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a246817-ee1c-404b-81c4-caf6a6a62581 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:12.401341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a246817-ee1c-404b-81c4-caf6a6a62581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:12.421876 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-4a246817-ee1c-404b-81c4-caf6a6a62581 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:12.423108 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-4a246817-ee1c-404b-81c4-caf6a6a62581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:12.431136 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 777/1556] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:12.432280 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726aae2-21bf-4fd2-bed0-4202c82a8edb None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:12.432557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726aae2-21bf-4fd2-bed0-4202c82a8edb None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:12.432833 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726aae2-21bf-4fd2-bed0-4202c82a8edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:12.453143 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-0726aae2-21bf-4fd2-bed0-4202c82a8edb None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:12.459788 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-0726aae2-21bf-4fd2-bed0-4202c82a8edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:12.462264 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 778/1557] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:12.463049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0726aae2-21bf-4fd2-bed0-4202c82a8edb None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:12.474228 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 780/1558] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:03:12 2022] POST /identity/v3/users => generated 288 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:12.474228 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2841a6b-05ca-4417-bfd3-8eb254be0201 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:12.474228 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2841a6b-05ca-4417-bfd3-8eb254be0201 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:12.474228 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2841a6b-05ca-4417-bfd3-8eb254be0201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:12.475849 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0726aae2-21bf-4fd2-bed0-4202c82a8edb None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:12.481267 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c97f35-b165-413b-a0e9-c1a1b8095bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:12.481878 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c97f35-b165-413b-a0e9-c1a1b8095bec admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:12.482153 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c97f35-b165-413b-a0e9-c1a1b8095bec admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:12.482515 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c97f35-b165-413b-a0e9-c1a1b8095bec admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:12.483218 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c97f35-b165-413b-a0e9-c1a1b8095bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:12.484761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c97f35-b165-413b-a0e9-c1a1b8095bec admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:12.494199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-a2841a6b-05ca-4417-bfd3-8eb254be0201 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:12.498821 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 779/1559] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:12.498937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a2841a6b-05ca-4417-bfd3-8eb254be0201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:12.499041 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c97f35-b165-413b-a0e9-c1a1b8095bec admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:12.506929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c97f35-b165-413b-a0e9-c1a1b8095bec admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:12.507166 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 781/1560] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:12.512649 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb5607-b950-4e7e-b9c1-36c1cb1ffe4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:12.512814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c80094-f372-46d6-a362-dc91a8a9cc82 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:12.512975 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c80094-f372-46d6-a362-dc91a8a9cc82 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:12.513242 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb5607-b950-4e7e-b9c1-36c1cb1ffe4d admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:12.513357 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c80094-f372-46d6-a362-dc91a8a9cc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:12.513521 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb5607-b950-4e7e-b9c1-36c1cb1ffe4d admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:12.513793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb5607-b950-4e7e-b9c1-36c1cb1ffe4d admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:12.514440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cb5607-b950-4e7e-b9c1-36c1cb1ffe4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:12.516434 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cb5607-b950-4e7e-b9c1-36c1cb1ffe4d admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:12.533164 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-14c80094-f372-46d6-a362-dc91a8a9cc82 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:12.534354 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-14c80094-f372-46d6-a362-dc91a8a9cc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:12.542231 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 782/1561] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:12.550203 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d2b8d-cdb2-472e-8827-00f460dc48fa None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:12.550203 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d2b8d-cdb2-472e-8827-00f460dc48fa None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:12.550203 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d2b8d-cdb2-472e-8827-00f460dc48fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:12.563959 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-4d7d2b8d-cdb2-472e-8827-00f460dc48fa None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:12.565190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-4d7d2b8d-cdb2-472e-8827-00f460dc48fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:12.573229 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 783/1562] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:12.575471 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576bff1-2bcc-4271-98be-f52eae409298 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:12.575752 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576bff1-2bcc-4271-98be-f52eae409298 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:12.576028 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576bff1-2bcc-4271-98be-f52eae409298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:12.595317 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-2576bff1-2bcc-4271-98be-f52eae409298 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:12.597478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2576bff1-2bcc-4271-98be-f52eae409298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:12.604634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 784/1563] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:12.605701 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576bff1-2bcc-4271-98be-f52eae409298 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:12.614988 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576bff1-2bcc-4271-98be-f52eae409298 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:12.619723 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25bd80-52fd-495a-abc5-e4dec73ae213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:12.620901 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25bd80-52fd-495a-abc5-e4dec73ae213 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:12.621378 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25bd80-52fd-495a-abc5-e4dec73ae213 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:12.621775 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25bd80-52fd-495a-abc5-e4dec73ae213 admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:12.622488 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa25bd80-52fd-495a-abc5-e4dec73ae213 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:03:12.622792 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa25bd80-52fd-495a-abc5-e4dec73ae213 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:12.632301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa25bd80-52fd-495a-abc5-e4dec73ae213 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:12.632454 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 780/1564] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:03:12 2022] POST /identity/v3/projects => generated 365 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:12.632454 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb5607-b950-4e7e-b9c1-36c1cb1ffe4d admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:12.644641 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 785/1565] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:03:12.644641 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25bd80-52fd-495a-abc5-e4dec73ae213 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:12.648255 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25bd80-52fd-495a-abc5-e4dec73ae213 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:12.660178 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:12.688594 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:12.705412 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f1251-d337-4e1a-9ee5-ad43e76555b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:12.706215 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f1251-d337-4e1a-9ee5-ad43e76555b3 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:12.706542 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f1251-d337-4e1a-9ee5-ad43e76555b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:12.706820 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f1251-d337-4e1a-9ee5-ad43e76555b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:12.707472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f1251-d337-4e1a-9ee5-ad43e76555b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:12.709197 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f1251-d337-4e1a-9ee5-ad43e76555b3 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:12.710440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb5607-b950-4e7e-b9c1-36c1cb1ffe4d admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:12.715376 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9910a2-e9df-46c4-af25-b03e486d882a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:12.724936 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9910a2-e9df-46c4-af25-b03e486d882a admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:12.724936 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9910a2-e9df-46c4-af25-b03e486d882a admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:12.724936 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9910a2-e9df-46c4-af25-b03e486d882a admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:12.724936 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9910a2-e9df-46c4-af25-b03e486d882a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:12.724936 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9910a2-e9df-46c4-af25-b03e486d882a admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:12.726033 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 786/1566] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:12.726033 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f1251-d337-4e1a-9ee5-ad43e76555b3 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:12.730710 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 781/1567] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:12.730710 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9910a2-e9df-46c4-af25-b03e486d882a admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:12.731580 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f1251-d337-4e1a-9ee5-ad43e76555b3 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:12.736268 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6415d-9528-484d-b1b6-d39fab2903d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:12.736800 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6415d-9528-484d-b1b6-d39fab2903d4 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:12.737075 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6415d-9528-484d-b1b6-d39fab2903d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:12.737377 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6415d-9528-484d-b1b6-d39fab2903d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:12.737974 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e6415d-9528-484d-b1b6-d39fab2903d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:03:12.738270 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e6415d-9528-484d-b1b6-d39fab2903d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:12.739880 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e6415d-9528-484d-b1b6-d39fab2903d4 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:12.741779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9910a2-e9df-46c4-af25-b03e486d882a admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:12.747505 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb8850-14b7-4bb3-97d8-d26e31f6e3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:12.748251 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb8850-14b7-4bb3-97d8-d26e31f6e3ee admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:12.748535 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb8850-14b7-4bb3-97d8-d26e31f6e3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:12.748806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb8850-14b7-4bb3-97d8-d26e31f6e3ee admin admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:12.749483 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb8850-14b7-4bb3-97d8-d26e31f6e3ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:03:12.749788 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb8850-14b7-4bb3-97d8-d26e31f6e3ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:12.751410 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb8850-14b7-4bb3-97d8-d26e31f6e3ee admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:12.758578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 787/1568] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:03:12.758578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6415d-9528-484d-b1b6-d39fab2903d4 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:12.773202 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 782/1569] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:03:12.773202 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb8850-14b7-4bb3-97d8-d26e31f6e3ee admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:12.782695 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6415d-9528-484d-b1b6-d39fab2903d4 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:12.782818 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb8850-14b7-4bb3-97d8-d26e31f6e3ee admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:12.782818 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4e6f8-1c92-40c4-b7cc-d6c91644ba80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:12.783030 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a67b8-04cd-4acc-a5a5-cf1424736b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:12.783132 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4e6f8-1c92-40c4-b7cc-d6c91644ba80 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:12.783132 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4e6f8-1c92-40c4-b7cc-d6c91644ba80 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:12.783132 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4e6f8-1c92-40c4-b7cc-d6c91644ba80 admin admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:12.783583 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d4e6f8-1c92-40c4-b7cc-d6c91644ba80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:03:12.783884 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d4e6f8-1c92-40c4-b7cc-d6c91644ba80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:12.784365 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a67b8-04cd-4acc-a5a5-cf1424736b29 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:12.784657 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a67b8-04cd-4acc-a5a5-cf1424736b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:12.784934 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a67b8-04cd-4acc-a5a5-cf1424736b29 admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:12.785510 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d4e6f8-1c92-40c4-b7cc-d6c91644ba80 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:12.785636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51a67b8-04cd-4acc-a5a5-cf1424736b29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:03:12.785917 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51a67b8-04cd-4acc-a5a5-cf1424736b29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:12.787555 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51a67b8-04cd-4acc-a5a5-cf1424736b29 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:12.797889 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 783/1570] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:03:12.798728 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4e6f8-1c92-40c4-b7cc-d6c91644ba80 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:12.807446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 788/1571] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:03:12.809087 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a67b8-04cd-4acc-a5a5-cf1424736b29 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:12.809869 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4e6f8-1c92-40c4-b7cc-d6c91644ba80 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:12.814663 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2bd7a-29c3-4a7b-95b8-e788b23f1293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:12.815265 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2bd7a-29c3-4a7b-95b8-e788b23f1293 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:12.816593 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2bd7a-29c3-4a7b-95b8-e788b23f1293 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:12.816593 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2bd7a-29c3-4a7b-95b8-e788b23f1293 admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:12.816593 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d2bd7a-29c3-4a7b-95b8-e788b23f1293 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:12.817400 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: WARNING py.warnings [None req-f2d2bd7a-29c3-4a7b-95b8-e788b23f1293 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:03:12.817400 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: warnings.warn(msg) Aug 31 11:03:12.817400 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: Aug 31 11:03:12.818629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d2bd7a-29c3-4a7b-95b8-e788b23f1293 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:12.819516 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a67b8-04cd-4acc-a5a5-cf1424736b29 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:12.824908 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05133e-39a8-488d-b437-1a590b31b2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:12.825497 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05133e-39a8-488d-b437-1a590b31b2f6 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:12.825779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05133e-39a8-488d-b437-1a590b31b2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:12.826058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05133e-39a8-488d-b437-1a590b31b2f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:12.826719 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d05133e-39a8-488d-b437-1a590b31b2f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:12.828721 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d05133e-39a8-488d-b437-1a590b31b2f6 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:13.015924 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 784/1572] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:03:12 2022] POST /identity/v3/users => generated 283 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:13.016117 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 789/1573] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:03:12 2022] POST /identity/v3/projects => generated 379 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:13.016215 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2bd7a-29c3-4a7b-95b8-e788b23f1293 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:13.016358 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05133e-39a8-488d-b437-1a590b31b2f6 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:13.024430 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2bd7a-29c3-4a7b-95b8-e788b23f1293 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:13.078746 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05133e-39a8-488d-b437-1a590b31b2f6 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:13.083187 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34e62f-957e-47b7-8192-2f72d2be8d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:13.083403 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243038fd-a5d2-4e82-89fb-b1a7fa47ae19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:13.083901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34e62f-957e-47b7-8192-2f72d2be8d79 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:13.084002 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243038fd-a5d2-4e82-89fb-b1a7fa47ae19 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:13.084179 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34e62f-957e-47b7-8192-2f72d2be8d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:13.084274 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243038fd-a5d2-4e82-89fb-b1a7fa47ae19 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:13.084475 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34e62f-957e-47b7-8192-2f72d2be8d79 admin admin] PATH_INFO: `/v3/projects/1d62244979394d21a86fdbbe6a56ad36/users/d5d0010f584249b4a22fd31634c9fd8b/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:13.084588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243038fd-a5d2-4e82-89fb-b1a7fa47ae19 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:13.085131 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243038fd-a5d2-4e82-89fb-b1a7fa47ae19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:13.086755 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243038fd-a5d2-4e82-89fb-b1a7fa47ae19 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:13.099969 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 790/1574] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:13.099969 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243038fd-a5d2-4e82-89fb-b1a7fa47ae19 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:13.100258 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34e62f-957e-47b7-8192-2f72d2be8d79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d62244979394d21a86fdbbe6a56ad36, user_id=d5d0010f584249b4a22fd31634c9fd8b, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:13.107076 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34e62f-957e-47b7-8192-2f72d2be8d79 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:13.110009 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243038fd-a5d2-4e82-89fb-b1a7fa47ae19 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:13.114734 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e11c45-534e-4d35-bdb5-3130d796f0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:13.120409 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e11c45-534e-4d35-bdb5-3130d796f0bc admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:13.120409 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e11c45-534e-4d35-bdb5-3130d796f0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:13.120409 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e11c45-534e-4d35-bdb5-3130d796f0bc admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:13.120409 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e11c45-534e-4d35-bdb5-3130d796f0bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:13.120409 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e11c45-534e-4d35-bdb5-3130d796f0bc admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:13.207053 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 785/1575] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:13 2022] PUT /identity/v3/projects/1d62244979394d21a86fdbbe6a56ad36/users/d5d0010f584249b4a22fd31634c9fd8b/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:13.207861 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34e62f-957e-47b7-8192-2f72d2be8d79 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:13.240662 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 791/1576] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:03:13 2022] POST /identity/v3/projects => generated 363 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:13.240662 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e11c45-534e-4d35-bdb5-3130d796f0bc admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:13.272393 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34e62f-957e-47b7-8192-2f72d2be8d79 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:13.281304 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:13.291505 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:13.302705 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e11c45-534e-4d35-bdb5-3130d796f0bc admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:13.307684 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c8bd4-6911-4b3a-ad4b-a6a6a633b99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:13.308339 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c8bd4-6911-4b3a-ad4b-a6a6a633b99a admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:13.308611 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c8bd4-6911-4b3a-ad4b-a6a6a633b99a admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:13.308877 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c8bd4-6911-4b3a-ad4b-a6a6a633b99a admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:13.309540 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c8bd4-6911-4b3a-ad4b-a6a6a633b99a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:13.311469 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c8bd4-6911-4b3a-ad4b-a6a6a633b99a admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:13.332032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e143ae-fcf9-4859-83db-42f53159350c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:13.332646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e143ae-fcf9-4859-83db-42f53159350c admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:13.332925 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e143ae-fcf9-4859-83db-42f53159350c admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:13.333198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e143ae-fcf9-4859-83db-42f53159350c admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:13.333862 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e143ae-fcf9-4859-83db-42f53159350c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:13.335889 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e143ae-fcf9-4859-83db-42f53159350c admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:13.893633 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 792/1577] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:03:13 2022] POST /identity/v3/projects => generated 373 bytes in 653 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:13.893633 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c8bd4-6911-4b3a-ad4b-a6a6a633b99a admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:13.951640 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c8bd4-6911-4b3a-ad4b-a6a6a633b99a admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:13.956165 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f72ea-3b59-44f5-af87-1120321fd356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:13.956767 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f72ea-3b59-44f5-af87-1120321fd356 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:13.957044 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f72ea-3b59-44f5-af87-1120321fd356 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:13.957315 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f72ea-3b59-44f5-af87-1120321fd356 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:13.957946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f72ea-3b59-44f5-af87-1120321fd356 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:13.959886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f72ea-3b59-44f5-af87-1120321fd356 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:13.999271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 786/1578] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:03:13 2022] POST /identity/v3/projects => generated 405 bytes in 792 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:14.006097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e143ae-fcf9-4859-83db-42f53159350c admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:14.061865 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e143ae-fcf9-4859-83db-42f53159350c admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:14.070371 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7133d6c0-e893-4724-829a-de9c45cdd14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:14.071021 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133d6c0-e893-4724-829a-de9c45cdd14d admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:14.071299 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133d6c0-e893-4724-829a-de9c45cdd14d admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:14.071569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133d6c0-e893-4724-829a-de9c45cdd14d admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:14.072211 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7133d6c0-e893-4724-829a-de9c45cdd14d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:14.074810 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7133d6c0-e893-4724-829a-de9c45cdd14d admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:14.089756 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 787/1579] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:14.090593 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7133d6c0-e893-4724-829a-de9c45cdd14d admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:14.100548 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7133d6c0-e893-4724-829a-de9c45cdd14d admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:14.106283 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ea1fa-2c67-451e-9326-dc4a035289ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:14.106938 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ea1fa-2c67-451e-9326-dc4a035289ae admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:14.107218 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ea1fa-2c67-451e-9326-dc4a035289ae admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:14.107617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ea1fa-2c67-451e-9326-dc4a035289ae admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:14.108204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116ea1fa-2c67-451e-9326-dc4a035289ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:14.110128 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116ea1fa-2c67-451e-9326-dc4a035289ae admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:14.140524 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 793/1580] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:03:13 2022] POST /identity/v3/projects => generated 393 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:14.141383 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f72ea-3b59-44f5-af87-1120321fd356 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:14.201952 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f72ea-3b59-44f5-af87-1120321fd356 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:14.206843 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cf3e6-4392-47f6-94d0-31c46232c7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:14.207495 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cf3e6-4392-47f6-94d0-31c46232c7bc admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:14.207776 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cf3e6-4392-47f6-94d0-31c46232c7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:14.208047 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cf3e6-4392-47f6-94d0-31c46232c7bc admin admin] PATH_INFO: `/v3/projects/58687b56ab864698b5c107cbea52a8dc/users/5d1ce9150ba543b1b06f8edb5a545af0/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:14.216085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 788/1581] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:03:14 2022] POST /identity/v3/users => generated 290 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:14.216085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a1826-da72-4845-a82f-335b9963a560 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:14.216085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a1826-da72-4845-a82f-335b9963a560 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:14.216085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a1826-da72-4845-a82f-335b9963a560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:14.223048 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5cf3e6-4392-47f6-94d0-31c46232c7bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=58687b56ab864698b5c107cbea52a8dc, user_id=5d1ce9150ba543b1b06f8edb5a545af0, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:14.224108 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: WARNING py.warnings [None req-bc5cf3e6-4392-47f6-94d0-31c46232c7bc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:03:14.224108 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: warnings.warn(msg) Aug 31 11:03:14.224108 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: Aug 31 11:03:14.226067 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5cf3e6-4392-47f6-94d0-31c46232c7bc admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:14.234239 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-ee2a1826-da72-4845-a82f-335b9963a560 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:14.235503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-ee2a1826-da72-4845-a82f-335b9963a560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:14.243126 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 789/1582] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:14.243855 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a1826-da72-4845-a82f-335b9963a560 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:14.253892 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a1826-da72-4845-a82f-335b9963a560 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:14.258748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811cd20-f9a8-418e-89f3-95741ff16ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:14.259340 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811cd20-f9a8-418e-89f3-95741ff16ce0 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:14.259617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811cd20-f9a8-418e-89f3-95741ff16ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:14.259892 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811cd20-f9a8-418e-89f3-95741ff16ce0 admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:14.260538 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b811cd20-f9a8-418e-89f3-95741ff16ce0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:14.263257 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b811cd20-f9a8-418e-89f3-95741ff16ce0 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:14.316766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 794/1583] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:14 2022] PUT /identity/v3/projects/58687b56ab864698b5c107cbea52a8dc/users/5d1ce9150ba543b1b06f8edb5a545af0/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:14.317877 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cf3e6-4392-47f6-94d0-31c46232c7bc admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:14.351862 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:14.366594 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 790/1584] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:03:14 2022] POST /identity/v3/users => generated 282 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:14.368736 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811cd20-f9a8-418e-89f3-95741ff16ce0 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:14.379086 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811cd20-f9a8-418e-89f3-95741ff16ce0 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:14.381738 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cf3e6-4392-47f6-94d0-31c46232c7bc admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:14.383931 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acaeca7-905a-45b0-9333-cb8f0789ef6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:14.384534 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acaeca7-905a-45b0-9333-cb8f0789ef6f admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:14.384810 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acaeca7-905a-45b0-9333-cb8f0789ef6f admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:14.385127 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acaeca7-905a-45b0-9333-cb8f0789ef6f admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:14.385748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acaeca7-905a-45b0-9333-cb8f0789ef6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:14.386584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799eb60c-4faa-4090-8311-55e7f04d9ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:14.387231 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799eb60c-4faa-4090-8311-55e7f04d9ad8 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:14.387575 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799eb60c-4faa-4090-8311-55e7f04d9ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:14.387818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799eb60c-4faa-4090-8311-55e7f04d9ad8 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:14.388395 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acaeca7-905a-45b0-9333-cb8f0789ef6f admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:14.388795 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799eb60c-4faa-4090-8311-55e7f04d9ad8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:14.391114 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799eb60c-4faa-4090-8311-55e7f04d9ad8 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:14.500731 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 791/1585] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:03:14 2022] POST /identity/v3/users => generated 303 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:14.501481 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acaeca7-905a-45b0-9333-cb8f0789ef6f admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:14.511627 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acaeca7-905a-45b0-9333-cb8f0789ef6f admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:14.516394 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca38e1b-60f8-43a8-963e-152bf44e1e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:14.516989 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca38e1b-60f8-43a8-963e-152bf44e1e2d admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:14.517354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca38e1b-60f8-43a8-963e-152bf44e1e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:14.517617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca38e1b-60f8-43a8-963e-152bf44e1e2d admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:14.518303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca38e1b-60f8-43a8-963e-152bf44e1e2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:14.520084 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca38e1b-60f8-43a8-963e-152bf44e1e2d admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:14.533843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 792/1586] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:14.533843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca38e1b-60f8-43a8-963e-152bf44e1e2d admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:14.534141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 795/1587] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:03:14 2022] POST /identity/v3/users => generated 287 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:14.534141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799eb60c-4faa-4090-8311-55e7f04d9ad8 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:14.544524 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799eb60c-4faa-4090-8311-55e7f04d9ad8 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:14.545771 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca38e1b-60f8-43a8-963e-152bf44e1e2d admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:14.549183 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc65d5-f3f8-4ce4-b842-13b049870d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:14.549779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc65d5-f3f8-4ce4-b842-13b049870d6c admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:14.550052 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc65d5-f3f8-4ce4-b842-13b049870d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:14.550322 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc65d5-f3f8-4ce4-b842-13b049870d6c admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:14.550788 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43498c9-cf81-45bc-b9a5-7fc46b84b36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:14.551061 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc65d5-f3f8-4ce4-b842-13b049870d6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:14.551386 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43498c9-cf81-45bc-b9a5-7fc46b84b36e admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:14.551669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43498c9-cf81-45bc-b9a5-7fc46b84b36e admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:14.551941 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43498c9-cf81-45bc-b9a5-7fc46b84b36e admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:14.552566 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc65d5-f3f8-4ce4-b842-13b049870d6c admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:14.552697 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43498c9-cf81-45bc-b9a5-7fc46b84b36e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:14.554511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43498c9-cf81-45bc-b9a5-7fc46b84b36e admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:14.564143 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 796/1588] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:14.569935 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc65d5-f3f8-4ce4-b842-13b049870d6c admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:14.569935 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-71fc65d5-f3f8-4ce4-b842-13b049870d6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:14.624718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc65d5-f3f8-4ce4-b842-13b049870d6c admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:14.628379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4680e3-d634-420e-bba3-b30682beb080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:14.628979 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4680e3-d634-420e-bba3-b30682beb080 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:14.629254 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4680e3-d634-420e-bba3-b30682beb080 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:14.629523 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4680e3-d634-420e-bba3-b30682beb080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:14.638597 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4680e3-d634-420e-bba3-b30682beb080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:14.640027 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4680e3-d634-420e-bba3-b30682beb080 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:14.649297 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 797/1589] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:14.650488 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30a5c4-5ee8-4f77-8fd2-9181dc48274a None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:14.650771 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30a5c4-5ee8-4f77-8fd2-9181dc48274a None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:14.651042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30a5c4-5ee8-4f77-8fd2-9181dc48274a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:14.666955 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 793/1590] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:03:14 2022] POST /identity/v3/users => generated 297 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:14.669047 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43498c9-cf81-45bc-b9a5-7fc46b84b36e admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:14.671404 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-ef30a5c4-5ee8-4f77-8fd2-9181dc48274a None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:14.672660 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-ef30a5c4-5ee8-4f77-8fd2-9181dc48274a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:14.679557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43498c9-cf81-45bc-b9a5-7fc46b84b36e admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:14.680537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 798/1591] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:14.681317 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30a5c4-5ee8-4f77-8fd2-9181dc48274a None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:14.688380 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeefa14-0074-40b7-8cc0-33c99237c241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:14.688995 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeefa14-0074-40b7-8cc0-33c99237c241 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:14.689298 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeefa14-0074-40b7-8cc0-33c99237c241 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:14.689542 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeefa14-0074-40b7-8cc0-33c99237c241 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:14.690221 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeefa14-0074-40b7-8cc0-33c99237c241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:14.691755 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30a5c4-5ee8-4f77-8fd2-9181dc48274a None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:14.692128 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeefa14-0074-40b7-8cc0-33c99237c241 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:14.696517 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b4b63-fbbd-4137-8457-78b54513d3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:14.697110 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b4b63-fbbd-4137-8457-78b54513d3f3 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:14.697387 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b4b63-fbbd-4137-8457-78b54513d3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:14.697659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b4b63-fbbd-4137-8457-78b54513d3f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:14.698297 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b4b63-fbbd-4137-8457-78b54513d3f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:14.700135 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b4b63-fbbd-4137-8457-78b54513d3f3 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:14.703357 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 794/1592] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:14.707510 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeefa14-0074-40b7-8cc0-33c99237c241 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:14.713084 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 799/1593] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:14.713084 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b4b63-fbbd-4137-8457-78b54513d3f3 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:14.713356 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:14.716433 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeefa14-0074-40b7-8cc0-33c99237c241 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:14.721159 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30c38a-2382-4b42-9b3a-3990d4d45ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:14.721417 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b4b63-fbbd-4137-8457-78b54513d3f3 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:14.721966 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30c38a-2382-4b42-9b3a-3990d4d45ba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:14.722267 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30c38a-2382-4b42-9b3a-3990d4d45ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:14.722554 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30c38a-2382-4b42-9b3a-3990d4d45ba3 admin admin] PATH_INFO: `/v3/projects/17643718ef274305a1c33bf09bedef88/users/d0f259a199404c70b66531c38361685c/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:14.726335 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969df64-94de-44e1-8e1e-17ceac74cdbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:14.726637 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969df64-94de-44e1-8e1e-17ceac74cdbe admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:14.726875 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969df64-94de-44e1-8e1e-17ceac74cdbe admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:14.727145 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969df64-94de-44e1-8e1e-17ceac74cdbe admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:14.727799 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d969df64-94de-44e1-8e1e-17ceac74cdbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:14.729702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d969df64-94de-44e1-8e1e-17ceac74cdbe admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:14.740291 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e30c38a-2382-4b42-9b3a-3990d4d45ba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17643718ef274305a1c33bf09bedef88, user_id=d0f259a199404c70b66531c38361685c, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:14.742058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 800/1594] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:14.742058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969df64-94de-44e1-8e1e-17ceac74cdbe admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:14.743231 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e30c38a-2382-4b42-9b3a-3990d4d45ba3 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:14.751621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969df64-94de-44e1-8e1e-17ceac74cdbe admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:14.756156 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6fd1fb-6d29-49a4-b09e-eb9ddbae3c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:14.756733 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fd1fb-6d29-49a4-b09e-eb9ddbae3c67 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:14.757006 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fd1fb-6d29-49a4-b09e-eb9ddbae3c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:14.757272 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fd1fb-6d29-49a4-b09e-eb9ddbae3c67 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:14.757896 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6fd1fb-6d29-49a4-b09e-eb9ddbae3c67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:14.759552 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6fd1fb-6d29-49a4-b09e-eb9ddbae3c67 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:14.769628 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 801/1595] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:14.770405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6fd1fb-6d29-49a4-b09e-eb9ddbae3c67 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:14.780460 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6fd1fb-6d29-49a4-b09e-eb9ddbae3c67 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:14.785231 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48ac65-794a-4ff1-a435-a4c5273ab555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:14.785881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48ac65-794a-4ff1-a435-a4c5273ab555 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:14.786188 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48ac65-794a-4ff1-a435-a4c5273ab555 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:14.786443 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48ac65-794a-4ff1-a435-a4c5273ab555 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:14.787114 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d48ac65-794a-4ff1-a435-a4c5273ab555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:14.788795 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d48ac65-794a-4ff1-a435-a4c5273ab555 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:14.799969 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 802/1596] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:14.800870 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48ac65-794a-4ff1-a435-a4c5273ab555 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:14.811725 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48ac65-794a-4ff1-a435-a4c5273ab555 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:14.815604 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a82099-848b-42b3-9203-6685c59e99fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:14.816226 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a82099-848b-42b3-9203-6685c59e99fb service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:14.816507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a82099-848b-42b3-9203-6685c59e99fb service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:14.816777 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a82099-848b-42b3-9203-6685c59e99fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:14.827176 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a82099-848b-42b3-9203-6685c59e99fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:14.830495 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a82099-848b-42b3-9203-6685c59e99fb service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:14.844656 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 795/1597] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:14 2022] PUT /identity/v3/projects/17643718ef274305a1c33bf09bedef88/users/d0f259a199404c70b66531c38361685c/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:14.844656 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30c38a-2382-4b42-9b3a-3990d4d45ba3 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:14.844958 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 803/1598] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:14.844958 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a82099-848b-42b3-9203-6685c59e99fb service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:14.853154 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a82099-848b-42b3-9203-6685c59e99fb service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:14.908815 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:14.920202 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30c38a-2382-4b42-9b3a-3990d4d45ba3 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:14.921393 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e78111-d50d-4ec4-9a4a-d0a1eb607a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:14.922005 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e78111-d50d-4ec4-9a4a-d0a1eb607a02 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:14.922365 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e78111-d50d-4ec4-9a4a-d0a1eb607a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:14.922588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e78111-d50d-4ec4-9a4a-d0a1eb607a02 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:14.923264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e78111-d50d-4ec4-9a4a-d0a1eb607a02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:14.924898 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e78111-d50d-4ec4-9a4a-d0a1eb607a02 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:14.932022 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5bef9-4316-43dd-8731-f2e8ae9c24fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:14.933702 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5bef9-4316-43dd-8731-f2e8ae9c24fc admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:14.934021 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5bef9-4316-43dd-8731-f2e8ae9c24fc admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:14.934458 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5bef9-4316-43dd-8731-f2e8ae9c24fc admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:14.935469 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf5bef9-4316-43dd-8731-f2e8ae9c24fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:14.937283 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf5bef9-4316-43dd-8731-f2e8ae9c24fc admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:14.938239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 804/1599] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:14.939207 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e78111-d50d-4ec4-9a4a-d0a1eb607a02 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:14.949310 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e78111-d50d-4ec4-9a4a-d0a1eb607a02 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:14.952971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 796/1600] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:14.953904 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5bef9-4316-43dd-8731-f2e8ae9c24fc admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:14.954853 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42e333-fc2f-4fb8-97e5-5776a13d6359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:14.955559 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42e333-fc2f-4fb8-97e5-5776a13d6359 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:14.955909 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42e333-fc2f-4fb8-97e5-5776a13d6359 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:14.956249 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42e333-fc2f-4fb8-97e5-5776a13d6359 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:14.956991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42e333-fc2f-4fb8-97e5-5776a13d6359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:14.958702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42e333-fc2f-4fb8-97e5-5776a13d6359 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:14.965858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5bef9-4316-43dd-8731-f2e8ae9c24fc admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:14.970431 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 805/1601] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:14.971264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42e333-fc2f-4fb8-97e5-5776a13d6359 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:14.973557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d121b1-ad0f-48dc-a42d-e198e31713d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:14.974575 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d121b1-ad0f-48dc-a42d-e198e31713d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:14.975054 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d121b1-ad0f-48dc-a42d-e198e31713d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:14.975500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d121b1-ad0f-48dc-a42d-e198e31713d2 admin admin] PATH_INFO: `/v3/projects/37d98454f0d04e9496623da6700329f8/users/d1c0bed0663c4afb8c59dd77be320905/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:14.996834 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42e333-fc2f-4fb8-97e5-5776a13d6359 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:15.000583 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d121b1-ad0f-48dc-a42d-e198e31713d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37d98454f0d04e9496623da6700329f8, user_id=d1c0bed0663c4afb8c59dd77be320905, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:15.006470 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d121b1-ad0f-48dc-a42d-e198e31713d2 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:15.013957 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70850c2-d851-4c64-8590-28429875e86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:15.014768 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70850c2-d851-4c64-8590-28429875e86f admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:15.015142 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70850c2-d851-4c64-8590-28429875e86f admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:15.015486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70850c2-d851-4c64-8590-28429875e86f admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:15.016421 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70850c2-d851-4c64-8590-28429875e86f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:15.018922 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70850c2-d851-4c64-8590-28429875e86f admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:15.045055 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 806/1602] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:15.046348 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a290d-e0aa-4de3-819d-f0d180f65275 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:15.046652 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a290d-e0aa-4de3-819d-f0d180f65275 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:15.046927 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a290d-e0aa-4de3-819d-f0d180f65275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:15.075814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-ed5a290d-e0aa-4de3-819d-f0d180f65275 None None] MFA Rules not processed for user `d0f259a199404c70b66531c38361685c`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:15.079700 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-ed5a290d-e0aa-4de3-819d-f0d180f65275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:15.153253 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 807/1603] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:15.154081 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a290d-e0aa-4de3-819d-f0d180f65275 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:15.164687 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a290d-e0aa-4de3-819d-f0d180f65275 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:15.169969 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d345b3b-11a6-48f9-bafe-ea765a33ed1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:15.170704 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d345b3b-11a6-48f9-bafe-ea765a33ed1a admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:15.171064 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d345b3b-11a6-48f9-bafe-ea765a33ed1a admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:15.171343 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d345b3b-11a6-48f9-bafe-ea765a33ed1a admin admin] PATH_INFO: `/v3/projects/f7a3570ed60040acb043931071a7db89/users/9497ff14b8974ea78de5c71775050e68/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:15.174168 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 797/1604] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:14 2022] PUT /identity/v3/projects/37d98454f0d04e9496623da6700329f8/users/d1c0bed0663c4afb8c59dd77be320905/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:15.176137 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d121b1-ad0f-48dc-a42d-e198e31713d2 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:15.188726 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d345b3b-11a6-48f9-bafe-ea765a33ed1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7a3570ed60040acb043931071a7db89, user_id=9497ff14b8974ea78de5c71775050e68, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:15.191455 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d345b3b-11a6-48f9-bafe-ea765a33ed1a admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:15.250838 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d121b1-ad0f-48dc-a42d-e198e31713d2 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:15.269338 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de2eda-4474-4621-9953-5660cec9e212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:15.270063 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de2eda-4474-4621-9953-5660cec9e212 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:15.270348 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de2eda-4474-4621-9953-5660cec9e212 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:15.270670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de2eda-4474-4621-9953-5660cec9e212 admin admin] PATH_INFO: `/v3/projects/1e94c7860db745de8276db1246009631/users/c32d0d3c06d34eb281b2072ceaf272de/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:15.288288 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 808/1605] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:15 2022] PUT /identity/v3/projects/f7a3570ed60040acb043931071a7db89/users/9497ff14b8974ea78de5c71775050e68/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:15.288997 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d345b3b-11a6-48f9-bafe-ea765a33ed1a admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:15.290971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de2eda-4474-4621-9953-5660cec9e212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e94c7860db745de8276db1246009631, user_id=c32d0d3c06d34eb281b2072ceaf272de, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:15.294059 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de2eda-4474-4621-9953-5660cec9e212 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:15.359567 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d345b3b-11a6-48f9-bafe-ea765a33ed1a admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:15.365794 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f4348b-9842-4d8b-912f-a845fb73a4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:15.366512 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4348b-9842-4d8b-912f-a845fb73a4a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:15.366797 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4348b-9842-4d8b-912f-a845fb73a4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:15.367074 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4348b-9842-4d8b-912f-a845fb73a4a1 admin admin] PATH_INFO: `/v3/projects/9c60378933db4f0ba7f63dc89f791ae0/users/5d8d5ff442f241dc9b7e0479f1a19cab/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:15.384080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f4348b-9842-4d8b-912f-a845fb73a4a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c60378933db4f0ba7f63dc89f791ae0, user_id=5d8d5ff442f241dc9b7e0479f1a19cab, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:15.386494 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 798/1606] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:15 2022] PUT /identity/v3/projects/1e94c7860db745de8276db1246009631/users/c32d0d3c06d34eb281b2072ceaf272de/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:15.386494 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de2eda-4474-4621-9953-5660cec9e212 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:15.387544 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f4348b-9842-4d8b-912f-a845fb73a4a1 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:15.424790 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:15.448037 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de2eda-4474-4621-9953-5660cec9e212 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:15.453962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f270cc-8302-4b8c-a20a-929d52d20008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:15.454666 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f270cc-8302-4b8c-a20a-929d52d20008 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:15.454943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f270cc-8302-4b8c-a20a-929d52d20008 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:15.455214 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f270cc-8302-4b8c-a20a-929d52d20008 admin admin] PATH_INFO: `/v3/projects/b4c01c0f214342aa88e4f71862604dd4/users/927bd33801e34715a78b58b268e28c4b/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:15.471262 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f270cc-8302-4b8c-a20a-929d52d20008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4c01c0f214342aa88e4f71862604dd4, user_id=927bd33801e34715a78b58b268e28c4b, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:15.474691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f270cc-8302-4b8c-a20a-929d52d20008 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:15.479804 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 809/1607] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:15 2022] PUT /identity/v3/projects/9c60378933db4f0ba7f63dc89f791ae0/users/5d8d5ff442f241dc9b7e0479f1a19cab/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:15.480962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f4348b-9842-4d8b-912f-a845fb73a4a1 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:15.506987 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:15.557455 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f4348b-9842-4d8b-912f-a845fb73a4a1 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:15.562611 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77eeb47-10e6-4997-8aca-a9ed6c7fb55b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:15.563234 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77eeb47-10e6-4997-8aca-a9ed6c7fb55b service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:15.563512 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77eeb47-10e6-4997-8aca-a9ed6c7fb55b service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:15.563775 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77eeb47-10e6-4997-8aca-a9ed6c7fb55b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:15.582853 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 799/1608] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:15 2022] PUT /identity/v3/projects/b4c01c0f214342aa88e4f71862604dd4/users/927bd33801e34715a78b58b268e28c4b/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:15.584303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacfe7c0-b1f3-45dc-b66d-b9069aa0c9df None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:15.584590 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacfe7c0-b1f3-45dc-b66d-b9069aa0c9df None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:15.584861 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacfe7c0-b1f3-45dc-b66d-b9069aa0c9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:15.609682 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-cacfe7c0-b1f3-45dc-b66d-b9069aa0c9df None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:15.610165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-cacfe7c0-b1f3-45dc-b66d-b9069aa0c9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:15.666557 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77eeb47-10e6-4997-8aca-a9ed6c7fb55b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:15.668909 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77eeb47-10e6-4997-8aca-a9ed6c7fb55b service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:15.678485 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 810/1609] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:15.679841 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df286151-b06d-4bc7-8daa-b112360367f7 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:15.683634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df286151-b06d-4bc7-8daa-b112360367f7 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:15.683634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df286151-b06d-4bc7-8daa-b112360367f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:15.683866 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 800/1610] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:15.683866 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c38715-59eb-48ab-8256-806cd64d8c63 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:15.683866 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c38715-59eb-48ab-8256-806cd64d8c63 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:15.683866 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c38715-59eb-48ab-8256-806cd64d8c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:15.705202 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-df286151-b06d-4bc7-8daa-b112360367f7 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:15.706693 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-df286151-b06d-4bc7-8daa-b112360367f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:15.715323 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 811/1611] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:15.716661 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a384a-bd2d-403c-ae37-df171b1493a2 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:15.716997 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a384a-bd2d-403c-ae37-df171b1493a2 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:15.717329 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a384a-bd2d-403c-ae37-df171b1493a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:15.719958 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-61c38715-59eb-48ab-8256-806cd64d8c63 None None] MFA Rules not processed for user `c32d0d3c06d34eb281b2072ceaf272de`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:15.720887 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-61c38715-59eb-48ab-8256-806cd64d8c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:15.762923 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-915a384a-bd2d-403c-ae37-df171b1493a2 None None] MFA Rules not processed for user `5d8d5ff442f241dc9b7e0479f1a19cab`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:15.764481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-915a384a-bd2d-403c-ae37-df171b1493a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:15.800041 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 801/1612] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:15.804635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df6efe-bd3b-43ad-829d-b5845c0ee29f None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:15.807736 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df6efe-bd3b-43ad-829d-b5845c0ee29f None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:15.810226 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df6efe-bd3b-43ad-829d-b5845c0ee29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:15.851293 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-c4df6efe-bd3b-43ad-829d-b5845c0ee29f None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:15.852956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-c4df6efe-bd3b-43ad-829d-b5845c0ee29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:15.867755 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 802/1613] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:15.868642 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df6efe-bd3b-43ad-829d-b5845c0ee29f None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:15.894343 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 812/1614] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:15.898520 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a384a-bd2d-403c-ae37-df171b1493a2 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:15.949534 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:15.971229 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df6efe-bd3b-43ad-829d-b5845c0ee29f None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:15.976268 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c392f5a4-592d-4123-8785-1d8132c1429b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:15.977446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:15.977975 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392f5a4-592d-4123-8785-1d8132c1429b service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:15.978065 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392f5a4-592d-4123-8785-1d8132c1429b service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:15.979065 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392f5a4-592d-4123-8785-1d8132c1429b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:15.983074 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a384a-bd2d-403c-ae37-df171b1493a2 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:15.986927 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de85ec2-7c75-49e7-a879-0c08fbf9ad15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:15.987570 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de85ec2-7c75-49e7-a879-0c08fbf9ad15 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:15.987848 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de85ec2-7c75-49e7-a879-0c08fbf9ad15 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:15.988401 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de85ec2-7c75-49e7-a879-0c08fbf9ad15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:15.991234 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c392f5a4-592d-4123-8785-1d8132c1429b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:15.992662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c392f5a4-592d-4123-8785-1d8132c1429b service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:15.999754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de85ec2-7c75-49e7-a879-0c08fbf9ad15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:16.001307 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de85ec2-7c75-49e7-a879-0c08fbf9ad15 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:16.003449 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 803/1615] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:16.004848 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c392f5a4-592d-4123-8785-1d8132c1429b service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:16.017325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c392f5a4-592d-4123-8785-1d8132c1429b service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:16.020017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 813/1616] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:16.020878 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de85ec2-7c75-49e7-a879-0c08fbf9ad15 service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:16.024910 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd2173-44c1-4d2c-8e14-df4a885f18b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:16.025619 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd2173-44c1-4d2c-8e14-df4a885f18b7 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:16.026012 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd2173-44c1-4d2c-8e14-df4a885f18b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:16.026681 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd2173-44c1-4d2c-8e14-df4a885f18b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:16.028652 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd2173-44c1-4d2c-8e14-df4a885f18b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:16.030614 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd2173-44c1-4d2c-8e14-df4a885f18b7 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:16.031649 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de85ec2-7c75-49e7-a879-0c08fbf9ad15 service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:16.036646 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a08131-3ff4-48ca-b6b1-b5cb0b5f423e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:16.037405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a08131-3ff4-48ca-b6b1-b5cb0b5f423e service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:16.037581 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a08131-3ff4-48ca-b6b1-b5cb0b5f423e service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:16.037855 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a08131-3ff4-48ca-b6b1-b5cb0b5f423e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:16.075504 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a08131-3ff4-48ca-b6b1-b5cb0b5f423e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:16.077403 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a08131-3ff4-48ca-b6b1-b5cb0b5f423e service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:16.087987 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 814/1617] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:16.090108 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b672a5-3f4d-4087-99dc-e4a08329e116 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:16.090424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b672a5-3f4d-4087-99dc-e4a08329e116 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:16.090683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b672a5-3f4d-4087-99dc-e4a08329e116 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:16.092093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 815/1618] 10.222.0.16 () {54 vars in 962 bytes} [Wed Aug 31 11:03:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:03:16.097743 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf6541-81b8-4564-a518-1396df58e4e8 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:16.098087 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf6541-81b8-4564-a518-1396df58e4e8 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:16.098319 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf6541-81b8-4564-a518-1396df58e4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:16.150738 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-6ebf6541-81b8-4564-a518-1396df58e4e8 None None] MFA Rules not processed for user `f11971a594014d86968c3356d11d42f7`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:16.152745 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-6ebf6541-81b8-4564-a518-1396df58e4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:16.200936 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 804/1619] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:03:16 2022] POST /identity/v3/projects => generated 363 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:16.205713 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd2173-44c1-4d2c-8e14-df4a885f18b7 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:16.242508 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 816/1620] 10.222.0.16 () {58 vars in 1059 bytes} [Wed Aug 31 11:03:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:16.252224 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf6541-81b8-4564-a518-1396df58e4e8 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:16.267327 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf6541-81b8-4564-a518-1396df58e4e8 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:16.318710 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd2173-44c1-4d2c-8e14-df4a885f18b7 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:16.327175 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b755bab-5c67-4ffa-bb78-6e7f6aff526d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:16.328036 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b755bab-5c67-4ffa-bb78-6e7f6aff526d admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:16.328470 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b755bab-5c67-4ffa-bb78-6e7f6aff526d admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:16.329326 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b755bab-5c67-4ffa-bb78-6e7f6aff526d admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:16.330076 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b755bab-5c67-4ffa-bb78-6e7f6aff526d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:16.344964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b755bab-5c67-4ffa-bb78-6e7f6aff526d admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:16.402972 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c819396-6e55-40ce-8e2d-9eed1fcae70a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:16.403605 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c819396-6e55-40ce-8e2d-9eed1fcae70a service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:16.403884 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c819396-6e55-40ce-8e2d-9eed1fcae70a service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:16.404281 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c819396-6e55-40ce-8e2d-9eed1fcae70a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:16.484647 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c819396-6e55-40ce-8e2d-9eed1fcae70a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:16.488577 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c819396-6e55-40ce-8e2d-9eed1fcae70a service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:16.499817 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 817/1621] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:03:16.559995 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:16.567872 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 805/1622] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:03:16 2022] POST /identity/v3/users => generated 288 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:16.571480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c819396-6e55-40ce-8e2d-9eed1fcae70a service nova] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:16.582514 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c819396-6e55-40ce-8e2d-9eed1fcae70a service nova] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:16.587353 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce89847-41f9-4ba6-8045-ee11ad2b8e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:16.588106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce89847-41f9-4ba6-8045-ee11ad2b8e93 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:16.588442 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce89847-41f9-4ba6-8045-ee11ad2b8e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:16.588776 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce89847-41f9-4ba6-8045-ee11ad2b8e93 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:16.589765 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce89847-41f9-4ba6-8045-ee11ad2b8e93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:16.591482 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce89847-41f9-4ba6-8045-ee11ad2b8e93 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:16.611645 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b755bab-5c67-4ffa-bb78-6e7f6aff526d admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:16.618871 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 818/1623] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:16.623307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b755bab-5c67-4ffa-bb78-6e7f6aff526d admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:16.631049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc25c6f-fa39-4c15-adb5-5dac2a3c6803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:16.631850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc25c6f-fa39-4c15-adb5-5dac2a3c6803 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:16.633692 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc25c6f-fa39-4c15-adb5-5dac2a3c6803 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:16.634064 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc25c6f-fa39-4c15-adb5-5dac2a3c6803 admin admin] PATH_INFO: `/v3/projects/0b740784a59a48b6bfdf0e9643a05dc8/users/3a830dface7643559a98b1ab9789361f/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:16.666007 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc25c6f-fa39-4c15-adb5-5dac2a3c6803 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b740784a59a48b6bfdf0e9643a05dc8, user_id=3a830dface7643559a98b1ab9789361f, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:16.671619 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc25c6f-fa39-4c15-adb5-5dac2a3c6803 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:16.817810 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a0628-2a56-4db3-9eff-cd0a3a4b1915 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:16.818098 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a0628-2a56-4db3-9eff-cd0a3a4b1915 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:16.818373 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a0628-2a56-4db3-9eff-cd0a3a4b1915 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:16.821826 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 819/1624] 10.222.0.16 () {54 vars in 912 bytes} [Wed Aug 31 11:03:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:03:16.827822 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eae4c2-ff40-4365-9900-b0435125f3fe None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:16.828108 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eae4c2-ff40-4365-9900-b0435125f3fe None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:16.828382 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eae4c2-ff40-4365-9900-b0435125f3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:16.858032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-56eae4c2-ff40-4365-9900-b0435125f3fe None None] MFA Rules not processed for user `f11971a594014d86968c3356d11d42f7`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:16.859336 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-56eae4c2-ff40-4365-9900-b0435125f3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:16.869344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 820/1625] 10.222.0.16 () {58 vars in 1009 bytes} [Wed Aug 31 11:03:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:16.896176 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eae4c2-ff40-4365-9900-b0435125f3fe None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:16.961066 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 806/1626] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:16 2022] PUT /identity/v3/projects/0b740784a59a48b6bfdf0e9643a05dc8/users/3a830dface7643559a98b1ab9789361f/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:16.963682 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc25c6f-fa39-4c15-adb5-5dac2a3c6803 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:17.004190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eae4c2-ff40-4365-9900-b0435125f3fe None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:17.110859 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc25c6f-fa39-4c15-adb5-5dac2a3c6803 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:17.116285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded85bd-d3ea-434b-b6a2-a02e1f86f6cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:17.116917 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded85bd-d3ea-434b-b6a2-a02e1f86f6cb service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:17.117196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded85bd-d3ea-434b-b6a2-a02e1f86f6cb service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:17.117473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded85bd-d3ea-434b-b6a2-a02e1f86f6cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:17.127516 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210ff5a-bcb0-44df-8afb-438be364723d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:17.128683 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210ff5a-bcb0-44df-8afb-438be364723d admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:17.129181 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210ff5a-bcb0-44df-8afb-438be364723d admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:17.129673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210ff5a-bcb0-44df-8afb-438be364723d admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:17.131117 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0210ff5a-bcb0-44df-8afb-438be364723d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:17.133130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0210ff5a-bcb0-44df-8afb-438be364723d admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:17.166036 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 807/1627] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:17.166977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210ff5a-bcb0-44df-8afb-438be364723d admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:17.200914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210ff5a-bcb0-44df-8afb-438be364723d admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:17.215673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8ae99-d67f-4b5e-bc2e-09b3e2015076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:17.216230 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8ae99-d67f-4b5e-bc2e-09b3e2015076 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:17.216596 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8ae99-d67f-4b5e-bc2e-09b3e2015076 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:17.216907 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8ae99-d67f-4b5e-bc2e-09b3e2015076 admin admin] PATH_INFO: `/v3/projects/0b740784a59a48b6bfdf0e9643a05dc8/users/3a830dface7643559a98b1ab9789361f/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:17.220071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8ae99-d67f-4b5e-bc2e-09b3e2015076 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b740784a59a48b6bfdf0e9643a05dc8, user_id=3a830dface7643559a98b1ab9789361f, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:17.226182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8ae99-d67f-4b5e-bc2e-09b3e2015076 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:17.262064 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 808/1628] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:17 2022] PUT /identity/v3/projects/0b740784a59a48b6bfdf0e9643a05dc8/users/3a830dface7643559a98b1ab9789361f/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:17.267561 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bd290-0ad2-4ad5-9544-9ad5cfdbc922 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:17.267561 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bd290-0ad2-4ad5-9544-9ad5cfdbc922 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:17.267561 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bd290-0ad2-4ad5-9544-9ad5cfdbc922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:17.271580 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded85bd-d3ea-434b-b6a2-a02e1f86f6cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:17.273039 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded85bd-d3ea-434b-b6a2-a02e1f86f6cb service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:17.298957 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:17.308471 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-0d4bd290-0ad2-4ad5-9544-9ad5cfdbc922 None None] MFA Rules not processed for user `3a830dface7643559a98b1ab9789361f`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:17.310015 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-0d4bd290-0ad2-4ad5-9544-9ad5cfdbc922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:17.312480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 821/1629] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:17.421770 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 809/1630] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:17 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:17.422919 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded85bd-d3ea-434b-b6a2-a02e1f86f6cb service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:17.442815 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded85bd-d3ea-434b-b6a2-a02e1f86f6cb service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:17.519135 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:17.548334 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c886528e-2522-4246-9129-15164cd6de92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:17.549053 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886528e-2522-4246-9129-15164cd6de92 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:17.549422 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886528e-2522-4246-9129-15164cd6de92 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:17.549791 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886528e-2522-4246-9129-15164cd6de92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:17.568800 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c886528e-2522-4246-9129-15164cd6de92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:17.571051 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c886528e-2522-4246-9129-15164cd6de92 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:17.586701 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 822/1631] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:17 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:03:17.990705 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4bd290-0ad2-4ad5-9544-9ad5cfdbc922 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:18.002915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4bd290-0ad2-4ad5-9544-9ad5cfdbc922 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:18.013595 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0331a5e8-0268-4d59-976d-a9c809ef39bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:18.014238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331a5e8-0268-4d59-976d-a9c809ef39bf service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:18.014537 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331a5e8-0268-4d59-976d-a9c809ef39bf service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:18.015012 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331a5e8-0268-4d59-976d-a9c809ef39bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:18.095671 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0331a5e8-0268-4d59-976d-a9c809ef39bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:18.097450 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0331a5e8-0268-4d59-976d-a9c809ef39bf service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:18.125186 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 810/1632] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:19.237062 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566088d-f71b-41d7-bec6-8aa6bc0d0e46 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:19.237838 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566088d-f71b-41d7-bec6-8aa6bc0d0e46 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:19.237838 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566088d-f71b-41d7-bec6-8aa6bc0d0e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:19.310254 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-2566088d-f71b-41d7-bec6-8aa6bc0d0e46 None None] MFA Rules not processed for user `9497ff14b8974ea78de5c71775050e68`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:19.316807 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2566088d-f71b-41d7-bec6-8aa6bc0d0e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:19.623596 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 823/1633] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:19 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:19.630203 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0331a5e8-0268-4d59-976d-a9c809ef39bf service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:19.639300 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958f92d-034f-4d50-8e23-c508271d4b84 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:19.644107 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958f92d-034f-4d50-8e23-c508271d4b84 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:19.644784 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958f92d-034f-4d50-8e23-c508271d4b84 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:19.647626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 824/1634] 10.222.0.16 () {54 vars in 962 bytes} [Wed Aug 31 11:03:19 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:03:19.674828 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca93e8-112e-4248-83bf-e87d1ceb6c0f None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:19.686842 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca93e8-112e-4248-83bf-e87d1ceb6c0f None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:19.689200 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca93e8-112e-4248-83bf-e87d1ceb6c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:19.767894 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-aaca93e8-112e-4248-83bf-e87d1ceb6c0f None None] MFA Rules not processed for user `f11971a594014d86968c3356d11d42f7`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:19.775384 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-aaca93e8-112e-4248-83bf-e87d1ceb6c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:19.909371 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0331a5e8-0268-4d59-976d-a9c809ef39bf service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:19.933596 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f87e8-bde1-46a3-b495-4000717061a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:19.934914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f87e8-bde1-46a3-b495-4000717061a1 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:19.935393 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f87e8-bde1-46a3-b495-4000717061a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:19.935862 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f87e8-bde1-46a3-b495-4000717061a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:19.943115 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f87e8-bde1-46a3-b495-4000717061a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:19.947071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f87e8-bde1-46a3-b495-4000717061a1 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:20.141972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 811/1635] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:03:19 2022] POST /identity/v3/projects => generated 405 bytes in 506 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:20.145156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f87e8-bde1-46a3-b495-4000717061a1 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:20.183876 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 825/1636] 10.222.0.16 () {58 vars in 1059 bytes} [Wed Aug 31 11:03:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 511 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:20.190888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b9092-18a5-4fde-8d50-4262c975a477 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:20.191754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b9092-18a5-4fde-8d50-4262c975a477 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:20.193297 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b9092-18a5-4fde-8d50-4262c975a477 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:20.197112 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 826/1637] 10.222.0.16 () {54 vars in 912 bytes} [Wed Aug 31 11:03:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:03:20.234592 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f970ae-54aa-4439-aa61-a0d193c0816e None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:20.235867 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f970ae-54aa-4439-aa61-a0d193c0816e None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:20.237354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:20.238553 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f970ae-54aa-4439-aa61-a0d193c0816e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:20.329076 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-a9f970ae-54aa-4439-aa61-a0d193c0816e None None] MFA Rules not processed for user `f11971a594014d86968c3356d11d42f7`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:20.332918 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a9f970ae-54aa-4439-aa61-a0d193c0816e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:20.455579 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f87e8-bde1-46a3-b495-4000717061a1 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:20.460740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e8125-26a1-410f-9d62-ee75a10dcdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:20.461371 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e8125-26a1-410f-9d62-ee75a10dcdce admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:20.461652 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e8125-26a1-410f-9d62-ee75a10dcdce admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:20.461926 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e8125-26a1-410f-9d62-ee75a10dcdce admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:20.475039 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8e8125-26a1-410f-9d62-ee75a10dcdce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:20.476964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8e8125-26a1-410f-9d62-ee75a10dcdce admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:20.519716 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:20.589145 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 827/1638] 10.222.0.16 () {58 vars in 1009 bytes} [Wed Aug 31 11:03:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:20.603821 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 812/1639] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:03:20 2022] POST /identity/v3/users => generated 309 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:20.608129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f970ae-54aa-4439-aa61-a0d193c0816e None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:20.626949 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f970ae-54aa-4439-aa61-a0d193c0816e None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:20.643094 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7efb37-6019-465e-9369-273e77e62a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:20.644503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7efb37-6019-465e-9369-273e77e62a6a admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:20.644946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7efb37-6019-465e-9369-273e77e62a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:20.645278 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7efb37-6019-465e-9369-273e77e62a6a admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:20.646061 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7efb37-6019-465e-9369-273e77e62a6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:20.647909 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7efb37-6019-465e-9369-273e77e62a6a admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:20.664059 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 828/1640] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:20.668426 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e8125-26a1-410f-9d62-ee75a10dcdce admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:20.684815 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e8125-26a1-410f-9d62-ee75a10dcdce admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:20.688386 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5e7b3-eeed-471b-922d-4a57eef74645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:20.689436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5e7b3-eeed-471b-922d-4a57eef74645 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:20.689850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5e7b3-eeed-471b-922d-4a57eef74645 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:20.690282 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5e7b3-eeed-471b-922d-4a57eef74645 admin admin] PATH_INFO: `/v3/projects/4902aae602384b9196f679ce77cfb4b8/users/e5e5c0bd424943789b1f18e26344c8ab/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:20.717055 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f5e7b3-eeed-471b-922d-4a57eef74645 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4902aae602384b9196f679ce77cfb4b8, user_id=e5e5c0bd424943789b1f18e26344c8ab, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:20.720234 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f5e7b3-eeed-471b-922d-4a57eef74645 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:20.751254 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7efb37-6019-465e-9369-273e77e62a6a admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:20.854252 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 813/1641] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:20 2022] PUT /identity/v3/projects/4902aae602384b9196f679ce77cfb4b8/users/e5e5c0bd424943789b1f18e26344c8ab/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:20.856622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5e7b3-eeed-471b-922d-4a57eef74645 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:20.871311 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7efb37-6019-465e-9369-273e77e62a6a admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:20.969729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ed777-2c99-4c79-9c12-0097b6d9397c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:20.970714 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ed777-2c99-4c79-9c12-0097b6d9397c service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:20.971134 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ed777-2c99-4c79-9c12-0097b6d9397c service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:20.971567 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ed777-2c99-4c79-9c12-0097b6d9397c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:21.016817 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5e7b3-eeed-471b-922d-4a57eef74645 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:21.024221 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012b687-c29a-4ac5-91d6-516760c4ae44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:21.025877 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012b687-c29a-4ac5-91d6-516760c4ae44 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:21.025877 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012b687-c29a-4ac5-91d6-516760c4ae44 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:21.025877 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012b687-c29a-4ac5-91d6-516760c4ae44 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:21.027467 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a012b687-c29a-4ac5-91d6-516760c4ae44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:21.029735 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a012b687-c29a-4ac5-91d6-516760c4ae44 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:21.046429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 814/1642] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:21.056662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012b687-c29a-4ac5-91d6-516760c4ae44 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:21.089489 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012b687-c29a-4ac5-91d6-516760c4ae44 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:21.103005 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5348a90-b195-48ab-a20c-6f83c27c3e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:21.106659 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5348a90-b195-48ab-a20c-6f83c27c3e7d admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:21.107119 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5348a90-b195-48ab-a20c-6f83c27c3e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:21.108222 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5348a90-b195-48ab-a20c-6f83c27c3e7d admin admin] PATH_INFO: `/v3/projects/4902aae602384b9196f679ce77cfb4b8/users/e5e5c0bd424943789b1f18e26344c8ab/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:21.112740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5348a90-b195-48ab-a20c-6f83c27c3e7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4902aae602384b9196f679ce77cfb4b8, user_id=e5e5c0bd424943789b1f18e26344c8ab, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:21.118982 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5348a90-b195-48ab-a20c-6f83c27c3e7d admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:21.150944 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 815/1643] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:21 2022] PUT /identity/v3/projects/4902aae602384b9196f679ce77cfb4b8/users/e5e5c0bd424943789b1f18e26344c8ab/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:21.169667 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ed777-2c99-4c79-9c12-0097b6d9397c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:21.171208 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ed777-2c99-4c79-9c12-0097b6d9397c service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:21.188470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 829/1644] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:24.302630 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5348a90-b195-48ab-a20c-6f83c27c3e7d admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:24.378935 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ed777-2c99-4c79-9c12-0097b6d9397c service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:24.415090 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ed777-2c99-4c79-9c12-0097b6d9397c service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:24.480448 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:24.522539 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39356f9-6f9f-4604-ae11-d48e64ebd12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:24.524723 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39356f9-6f9f-4604-ae11-d48e64ebd12c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:24.525009 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39356f9-6f9f-4604-ae11-d48e64ebd12c admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:24.525291 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39356f9-6f9f-4604-ae11-d48e64ebd12c admin admin] PATH_INFO: `/v3/users/c32d0d3c06d34eb281b2072ceaf272de` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:24.545056 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39356f9-6f9f-4604-ae11-d48e64ebd12c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c32d0d3c06d34eb281b2072ceaf272de)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:24.546184 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: WARNING py.warnings [None req-d39356f9-6f9f-4604-ae11-d48e64ebd12c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:03:24.546184 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: warnings.warn(msg) Aug 31 11:03:24.546184 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: Aug 31 11:03:24.552825 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39356f9-6f9f-4604-ae11-d48e64ebd12c admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:24.552977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5348a90-b195-48ab-a20c-6f83c27c3e7d admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:24.586787 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26759d-4763-449f-935e-e416b25a6c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:24.587438 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26759d-4763-449f-935e-e416b25a6c57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:24.587713 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26759d-4763-449f-935e-e416b25a6c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:24.588057 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26759d-4763-449f-935e-e416b25a6c57 admin admin] PATH_INFO: `/v3/users/5d8d5ff442f241dc9b7e0479f1a19cab` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:24.590026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26759d-4763-449f-935e-e416b25a6c57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d8d5ff442f241dc9b7e0479f1a19cab)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:24.592000 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26759d-4763-449f-935e-e416b25a6c57 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:24.632114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:25.285374 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-ca26759d-4763-449f-935e-e416b25a6c57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d8d5ff442f241dc9b7e0479f1a19cab'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:25.322530 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-d39356f9-6f9f-4604-ae11-d48e64ebd12c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c32d0d3c06d34eb281b2072ceaf272de'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:25.387685 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-d39356f9-6f9f-4604-ae11-d48e64ebd12c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c32d0d3c06d34eb281b2072ceaf272de'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:25.408969 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-d39356f9-6f9f-4604-ae11-d48e64ebd12c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c32d0d3c06d34eb281b2072ceaf272de'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:25.455350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-ca26759d-4763-449f-935e-e416b25a6c57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d8d5ff442f241dc9b7e0479f1a19cab'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:25.459874 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-ca26759d-4763-449f-935e-e416b25a6c57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d8d5ff442f241dc9b7e0479f1a19cab'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:25.459874 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-ca26759d-4763-449f-935e-e416b25a6c57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d8d5ff442f241dc9b7e0479f1a19cab'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:25.489528 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-ca26759d-4763-449f-935e-e416b25a6c57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d8d5ff442f241dc9b7e0479f1a19cab'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:25.523435 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 816/1645] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:03:24 2022] DELETE /identity/v3/users/5d8d5ff442f241dc9b7e0479f1a19cab => generated 0 bytes in 1222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:25.525104 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d366c5-6717-444f-b43c-219430158646 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:25.525521 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d366c5-6717-444f-b43c-219430158646 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:25.526115 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d366c5-6717-444f-b43c-219430158646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:25.580529 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-d39356f9-6f9f-4604-ae11-d48e64ebd12c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c32d0d3c06d34eb281b2072ceaf272de'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:25.581216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-d39356f9-6f9f-4604-ae11-d48e64ebd12c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c32d0d3c06d34eb281b2072ceaf272de'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:25.585493 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-b2d366c5-6717-444f-b43c-219430158646 None None] MFA Rules not processed for user `e5e5c0bd424943789b1f18e26344c8ab`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:25.586799 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-b2d366c5-6717-444f-b43c-219430158646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:25.592198 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 830/1646] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:03:24 2022] DELETE /identity/v3/users/c32d0d3c06d34eb281b2072ceaf272de => generated 0 bytes in 1214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:03:25.593742 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36abc8-b672-4aba-84be-cdd7b0862db8 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:25.594265 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36abc8-b672-4aba-84be-cdd7b0862db8 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:25.594586 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36abc8-b672-4aba-84be-cdd7b0862db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:25.657573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-2b36abc8-b672-4aba-84be-cdd7b0862db8 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:25.657573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2b36abc8-b672-4aba-84be-cdd7b0862db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:25.768623 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 817/1647] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:25 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:25.770175 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79057b2d-cb93-41ca-a180-39715e3b6ac1 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:25.770501 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79057b2d-cb93-41ca-a180-39715e3b6ac1 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:25.773811 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79057b2d-cb93-41ca-a180-39715e3b6ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:25.811581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-79057b2d-cb93-41ca-a180-39715e3b6ac1 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:25.813923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-79057b2d-cb93-41ca-a180-39715e3b6ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:25.858697 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 831/1648] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:25.954930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b36abc8-b672-4aba-84be-cdd7b0862db8 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:25.957045 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2b36abc8-b672-4aba-84be-cdd7b0862db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:25.997391 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 818/1649] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:26.133937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b36abc8-b672-4aba-84be-cdd7b0862db8 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:26.144793 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e3c62-f80a-4bdd-8dd9-03006844c97b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:26.145462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e3c62-f80a-4bdd-8dd9-03006844c97b service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:26.145737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e3c62-f80a-4bdd-8dd9-03006844c97b service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:26.146091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e3c62-f80a-4bdd-8dd9-03006844c97b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:26.171199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e3c62-f80a-4bdd-8dd9-03006844c97b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:26.175314 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e3c62-f80a-4bdd-8dd9-03006844c97b service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:26.190693 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:26.218025 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 832/1650] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:26.272028 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79057b2d-cb93-41ca-a180-39715e3b6ac1 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:26.273537 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-79057b2d-cb93-41ca-a180-39715e3b6ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:26.297156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79057b2d-cb93-41ca-a180-39715e3b6ac1 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:26.329583 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0a8ef-e19b-42d9-88a6-b0c54abeeff5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:26.335203 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0a8ef-e19b-42d9-88a6-b0c54abeeff5 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:26.335203 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0a8ef-e19b-42d9-88a6-b0c54abeeff5 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:26.335203 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0a8ef-e19b-42d9-88a6-b0c54abeeff5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:26.360060 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a0a8ef-e19b-42d9-88a6-b0c54abeeff5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:26.377660 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a0a8ef-e19b-42d9-88a6-b0c54abeeff5 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:26.425801 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 819/1651] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:26.807316 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:28.476835 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e3c62-f80a-4bdd-8dd9-03006844c97b service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:28.478296 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-7c8e3c62-f80a-4bdd-8dd9-03006844c97b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:28.498297 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e3c62-f80a-4bdd-8dd9-03006844c97b service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:28.516150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99866a70-3521-4d0c-963b-4604dea3e683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:28.517011 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99866a70-3521-4d0c-963b-4604dea3e683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:28.517427 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99866a70-3521-4d0c-963b-4604dea3e683 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:28.517841 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99866a70-3521-4d0c-963b-4604dea3e683 admin admin] PATH_INFO: `/v3/projects/9c60378933db4f0ba7f63dc89f791ae0` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:28.519306 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99866a70-3521-4d0c-963b-4604dea3e683 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c60378933db4f0ba7f63dc89f791ae0)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:28.521592 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99866a70-3521-4d0c-963b-4604dea3e683 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:28.715337 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-99866a70-3521-4d0c-963b-4604dea3e683 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c60378933db4f0ba7f63dc89f791ae0', 'request_id': 'req-99866a70-3521-4d0c-963b-4604dea3e683'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:28.818922 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929be22-4a78-4801-9a1d-1f112c64a90b None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:28.819461 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929be22-4a78-4801-9a1d-1f112c64a90b None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:28.819461 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929be22-4a78-4801-9a1d-1f112c64a90b None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:28.821227 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 820/1652] 10.222.0.16 () {54 vars in 921 bytes} [Wed Aug 31 11:03:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:03:28.869647 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-99866a70-3521-4d0c-963b-4604dea3e683 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c60378933db4f0ba7f63dc89f791ae0', 'request_id': 'req-99866a70-3521-4d0c-963b-4604dea3e683'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:28.892761 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-99866a70-3521-4d0c-963b-4604dea3e683 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c60378933db4f0ba7f63dc89f791ae0', 'request_id': 'req-99866a70-3521-4d0c-963b-4604dea3e683'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:28.945386 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-99866a70-3521-4d0c-963b-4604dea3e683 admin admin] The token cache is being invalidate because project 9c60378933db4f0ba7f63dc89f791ae0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:03:28.945386 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-99866a70-3521-4d0c-963b-4604dea3e683 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:28.949532 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-99866a70-3521-4d0c-963b-4604dea3e683 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:28.953644 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 833/1653] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:03:28 2022] DELETE /identity/v3/projects/9c60378933db4f0ba7f63dc89f791ae0 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:28.960087 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a565e9-997a-44f8-b302-2a2c5991761b None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:28.960368 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a565e9-997a-44f8-b302-2a2c5991761b None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:28.960640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a565e9-997a-44f8-b302-2a2c5991761b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:29.007754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-63a565e9-997a-44f8-b302-2a2c5991761b None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:29.009006 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-63a565e9-997a-44f8-b302-2a2c5991761b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:29.222647 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 821/1654] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:29.223742 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a6428-1650-439f-99c9-68614408adc6 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:29.224072 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a6428-1650-439f-99c9-68614408adc6 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:29.224351 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a6428-1650-439f-99c9-68614408adc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:29.257695 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-ec6a6428-1650-439f-99c9-68614408adc6 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:29.259002 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-ec6a6428-1650-439f-99c9-68614408adc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:29.286131 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 834/1655] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:29.286609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a565e9-997a-44f8-b302-2a2c5991761b None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:29.286794 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3cc14-e55f-4f4d-bd66-8c17deee0a59 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:29.286794 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3cc14-e55f-4f4d-bd66-8c17deee0a59 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:29.286794 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3cc14-e55f-4f4d-bd66-8c17deee0a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:29.304684 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a565e9-997a-44f8-b302-2a2c5991761b None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:29.336368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-72c3cc14-e55f-4f4d-bd66-8c17deee0a59 None None] MFA Rules not processed for user `f3db46a4b4e74cc0bb2a7bd6f0e99105`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:29.337599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-72c3cc14-e55f-4f4d-bd66-8c17deee0a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:29.351918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026d30a-609e-4f9c-80a8-16afab414ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:29.352779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026d30a-609e-4f9c-80a8-16afab414ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:29.353357 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026d30a-609e-4f9c-80a8-16afab414ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:29.358563 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026d30a-609e-4f9c-80a8-16afab414ef1 admin admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:29.359568 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d026d30a-609e-4f9c-80a8-16afab414ef1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:03:29.360098 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d026d30a-609e-4f9c-80a8-16afab414ef1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:29.362748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d026d30a-609e-4f9c-80a8-16afab414ef1 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:29.403114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 822/1656] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:03:29.411840 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026d30a-609e-4f9c-80a8-16afab414ef1 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:29.437248 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026d30a-609e-4f9c-80a8-16afab414ef1 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:29.451930 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce5a4f0-6d40-4e2c-be96-5ff81cdb8f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:29.452553 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5a4f0-6d40-4e2c-be96-5ff81cdb8f98 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:29.452831 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5a4f0-6d40-4e2c-be96-5ff81cdb8f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:29.453098 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5a4f0-6d40-4e2c-be96-5ff81cdb8f98 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:29.453750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce5a4f0-6d40-4e2c-be96-5ff81cdb8f98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:29.455676 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce5a4f0-6d40-4e2c-be96-5ff81cdb8f98 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:29.582493 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 835/1657] 10.222.0.16 () {58 vars in 1018 bytes} [Wed Aug 31 11:03:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:29.613844 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 823/1658] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:03:29 2022] POST /identity/v3/projects => generated 371 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:29.618269 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c3cc14-e55f-4f4d-bd66-8c17deee0a59 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:29.637941 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c3cc14-e55f-4f4d-bd66-8c17deee0a59 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:29.811168 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772882c2-448a-41bd-baaa-f0dca19fec44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:29.812266 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772882c2-448a-41bd-baaa-f0dca19fec44 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:29.812710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772882c2-448a-41bd-baaa-f0dca19fec44 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:29.813129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772882c2-448a-41bd-baaa-f0dca19fec44 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:29.826104 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772882c2-448a-41bd-baaa-f0dca19fec44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:29.827012 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce5a4f0-6d40-4e2c-be96-5ff81cdb8f98 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:29.829120 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-fce5a4f0-6d40-4e2c-be96-5ff81cdb8f98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:29.832335 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772882c2-448a-41bd-baaa-f0dca19fec44 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:29.939239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:29.951048 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 836/1659] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:03:29 2022] POST /identity/v3/users => generated 286 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:29.952005 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772882c2-448a-41bd-baaa-f0dca19fec44 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:29.991086 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772882c2-448a-41bd-baaa-f0dca19fec44 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:30.009703 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905da3c9-a9bd-4f7f-b76e-b8c2fdfcdbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:30.013262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905da3c9-a9bd-4f7f-b76e-b8c2fdfcdbb2 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:30.013262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905da3c9-a9bd-4f7f-b76e-b8c2fdfcdbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:30.013262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905da3c9-a9bd-4f7f-b76e-b8c2fdfcdbb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:30.013802 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905da3c9-a9bd-4f7f-b76e-b8c2fdfcdbb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:30.026446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905da3c9-a9bd-4f7f-b76e-b8c2fdfcdbb2 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:30.027153 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce5a4f0-6d40-4e2c-be96-5ff81cdb8f98 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:30.044502 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84bda6-e0d9-409f-a80d-db874812b9e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3db46a4b4e74cc0bb2a7bd6f0e99105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:30.047651 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84bda6-e0d9-409f-a80d-db874812b9e2 service placement] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:30.047651 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84bda6-e0d9-409f-a80d-db874812b9e2 service placement] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:30.047651 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84bda6-e0d9-409f-a80d-db874812b9e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:30.063266 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 837/1660] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:30.063608 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e84bda6-e0d9-409f-a80d-db874812b9e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:30.065347 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e84bda6-e0d9-409f-a80d-db874812b9e2 service placement] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:30.081288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 824/1661] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:30.082320 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84bda6-e0d9-409f-a80d-db874812b9e2 service placement] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:30.105667 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905da3c9-a9bd-4f7f-b76e-b8c2fdfcdbb2 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:30.122716 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84bda6-e0d9-409f-a80d-db874812b9e2 service placement] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:30.127356 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-905da3c9-a9bd-4f7f-b76e-b8c2fdfcdbb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:30.132178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e81f8-ea0a-46bc-97cf-8b2abb25c85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:30.133003 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e81f8-ea0a-46bc-97cf-8b2abb25c85e admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:30.133459 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e81f8-ea0a-46bc-97cf-8b2abb25c85e admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:30.133745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e81f8-ea0a-46bc-97cf-8b2abb25c85e admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:30.137355 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97e81f8-ea0a-46bc-97cf-8b2abb25c85e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:30.139274 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97e81f8-ea0a-46bc-97cf-8b2abb25c85e admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:30.155608 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 825/1662] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:30.159746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e81f8-ea0a-46bc-97cf-8b2abb25c85e admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:30.170515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905da3c9-a9bd-4f7f-b76e-b8c2fdfcdbb2 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:30.195253 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812502e-3eeb-4cdb-a174-edfe3a3bd57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:30.195444 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e81f8-ea0a-46bc-97cf-8b2abb25c85e admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:30.196347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812502e-3eeb-4cdb-a174-edfe3a3bd57f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:30.196633 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812502e-3eeb-4cdb-a174-edfe3a3bd57f admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:30.196912 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812502e-3eeb-4cdb-a174-edfe3a3bd57f admin admin] PATH_INFO: `/v3/projects/1e94c7860db745de8276db1246009631` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:30.198220 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2812502e-3eeb-4cdb-a174-edfe3a3bd57f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e94c7860db745de8276db1246009631)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:30.200212 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2812502e-3eeb-4cdb-a174-edfe3a3bd57f admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:30.217823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a2bad-8b13-4439-a553-e32df8c30de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:30.222633 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a2bad-8b13-4439-a553-e32df8c30de4 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:30.223105 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a2bad-8b13-4439-a553-e32df8c30de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:30.223950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a2bad-8b13-4439-a553-e32df8c30de4 admin admin] PATH_INFO: `/v3/projects/c661575302c34003a49a4aa85548563e/users/9b8a926f129b44749ce1d95f05af5357/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:30.261121 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a2bad-8b13-4439-a553-e32df8c30de4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c661575302c34003a49a4aa85548563e, user_id=9b8a926f129b44749ce1d95f05af5357, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:30.273931 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a2bad-8b13-4439-a553-e32df8c30de4 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:30.437320 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 826/1663] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:30 2022] PUT /identity/v3/projects/c661575302c34003a49a4aa85548563e/users/9b8a926f129b44749ce1d95f05af5357/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:30.438957 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd209ce-8a47-450d-b1fe-2aaae08c557f None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:30.439244 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd209ce-8a47-450d-b1fe-2aaae08c557f None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:30.439703 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd209ce-8a47-450d-b1fe-2aaae08c557f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:30.456750 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-2812502e-3eeb-4cdb-a174-edfe3a3bd57f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e94c7860db745de8276db1246009631', 'request_id': 'req-2812502e-3eeb-4cdb-a174-edfe3a3bd57f'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:30.514232 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-8dd209ce-8a47-450d-b1fe-2aaae08c557f None None] MFA Rules not processed for user `5d1ce9150ba543b1b06f8edb5a545af0`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:30.516482 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-8dd209ce-8a47-450d-b1fe-2aaae08c557f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:30.537686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:30.634928 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-2812502e-3eeb-4cdb-a174-edfe3a3bd57f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e94c7860db745de8276db1246009631', 'request_id': 'req-2812502e-3eeb-4cdb-a174-edfe3a3bd57f'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:30.647373 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-2812502e-3eeb-4cdb-a174-edfe3a3bd57f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e94c7860db745de8276db1246009631', 'request_id': 'req-2812502e-3eeb-4cdb-a174-edfe3a3bd57f'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:30.667409 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 827/1664] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:30.672348 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3349b9f-d50c-4638-bee8-1f53d84edc4a None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:30.672896 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3349b9f-d50c-4638-bee8-1f53d84edc4a None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:30.673228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3349b9f-d50c-4638-bee8-1f53d84edc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:30.685141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-2812502e-3eeb-4cdb-a174-edfe3a3bd57f admin admin] The token cache is being invalidate because project 1e94c7860db745de8276db1246009631 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:03:30.685969 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-2812502e-3eeb-4cdb-a174-edfe3a3bd57f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:30.688843 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-2812502e-3eeb-4cdb-a174-edfe3a3bd57f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:30.691187 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 838/1665] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:03:30 2022] DELETE /identity/v3/projects/1e94c7860db745de8276db1246009631 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:30.692678 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094afff-844f-4b30-9fc6-081d2c6074a3 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:30.693277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094afff-844f-4b30-9fc6-081d2c6074a3 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:30.694297 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094afff-844f-4b30-9fc6-081d2c6074a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:30.749027 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-e3349b9f-d50c-4638-bee8-1f53d84edc4a None None] MFA Rules not processed for user `9b8a926f129b44749ce1d95f05af5357`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:30.752359 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-e3349b9f-d50c-4638-bee8-1f53d84edc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:30.777891 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-d094afff-844f-4b30-9fc6-081d2c6074a3 None None] MFA Rules not processed for user `d5d0010f584249b4a22fd31634c9fd8b`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:30.779725 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-d094afff-844f-4b30-9fc6-081d2c6074a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:30.935365 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 839/1666] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:30.936679 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094afff-844f-4b30-9fc6-081d2c6074a3 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:30.938794 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-d094afff-844f-4b30-9fc6-081d2c6074a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:30.941984 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 828/1667] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:30.943627 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3349b9f-d50c-4638-bee8-1f53d84edc4a None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:30.945329 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-e3349b9f-d50c-4638-bee8-1f53d84edc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:31.076334 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094afff-844f-4b30-9fc6-081d2c6074a3 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:31.110918 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6136f21-a285-46db-ab73-1ecafd79d2c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:31.111633 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6136f21-a285-46db-ab73-1ecafd79d2c6 service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:31.111959 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6136f21-a285-46db-ab73-1ecafd79d2c6 service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:31.112242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6136f21-a285-46db-ab73-1ecafd79d2c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:31.119498 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-c6136f21-a285-46db-ab73-1ecafd79d2c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:31.155843 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6136f21-a285-46db-ab73-1ecafd79d2c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:31.156888 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3349b9f-d50c-4638-bee8-1f53d84edc4a None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:31.166026 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6136f21-a285-46db-ab73-1ecafd79d2c6 service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:31.191451 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adeaae1-3219-4064-8fb1-2ce08913cb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:31.191451 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adeaae1-3219-4064-8fb1-2ce08913cb59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:31.192656 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adeaae1-3219-4064-8fb1-2ce08913cb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:31.192947 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adeaae1-3219-4064-8fb1-2ce08913cb59 admin admin] PATH_INFO: `/v3/users/3a830dface7643559a98b1ab9789361f` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:31.199557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adeaae1-3219-4064-8fb1-2ce08913cb59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a830dface7643559a98b1ab9789361f)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:31.202137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 840/1668] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:31.203909 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6136f21-a285-46db-ab73-1ecafd79d2c6 service nova] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:31.205384 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-c6136f21-a285-46db-ab73-1ecafd79d2c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:31.209092 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adeaae1-3219-4064-8fb1-2ce08913cb59 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:31.223151 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6136f21-a285-46db-ab73-1ecafd79d2c6 service nova] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:31.229316 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71bc6f-0b62-4ca0-b8d1-521876326b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:31.230497 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71bc6f-0b62-4ca0-b8d1-521876326b36 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:31.230783 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71bc6f-0b62-4ca0-b8d1-521876326b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:31.231058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71bc6f-0b62-4ca0-b8d1-521876326b36 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:31.232228 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd71bc6f-0b62-4ca0-b8d1-521876326b36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:31.234220 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd71bc6f-0b62-4ca0-b8d1-521876326b36 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:31.348694 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 841/1669] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:03:31 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:31.349813 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71bc6f-0b62-4ca0-b8d1-521876326b36 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:31.352524 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-bd71bc6f-0b62-4ca0-b8d1-521876326b36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:31.534539 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71bc6f-0b62-4ca0-b8d1-521876326b36 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:31.561236 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae13125-9a9d-48af-b1ee-98a37eb744f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:31.561236 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae13125-9a9d-48af-b1ee-98a37eb744f2 service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:31.561236 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae13125-9a9d-48af-b1ee-98a37eb744f2 service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:31.561236 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae13125-9a9d-48af-b1ee-98a37eb744f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:31.629613 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-4adeaae1-3219-4064-8fb1-2ce08913cb59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a830dface7643559a98b1ab9789361f'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:31.730803 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae13125-9a9d-48af-b1ee-98a37eb744f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:31.732469 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae13125-9a9d-48af-b1ee-98a37eb744f2 service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:31.734193 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-4adeaae1-3219-4064-8fb1-2ce08913cb59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a830dface7643559a98b1ab9789361f'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:31.735043 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-4adeaae1-3219-4064-8fb1-2ce08913cb59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a830dface7643559a98b1ab9789361f'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:31.735871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-4adeaae1-3219-4064-8fb1-2ce08913cb59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a830dface7643559a98b1ab9789361f'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:31.786788 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-4adeaae1-3219-4064-8fb1-2ce08913cb59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a830dface7643559a98b1ab9789361f'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:31.815310 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 842/1670] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:31.815310 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae13125-9a9d-48af-b1ee-98a37eb744f2 service nova] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:31.828086 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 829/1671] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:03:30 2022] DELETE /identity/v3/users/3a830dface7643559a98b1ab9789361f => generated 0 bytes in 885 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:31.989628 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae13125-9a9d-48af-b1ee-98a37eb744f2 service nova] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:32.095935 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c85f7-f4bb-4a5a-b569-af94f3205c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:32.097022 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c85f7-f4bb-4a5a-b569-af94f3205c11 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:32.097292 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c85f7-f4bb-4a5a-b569-af94f3205c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:32.098343 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c85f7-f4bb-4a5a-b569-af94f3205c11 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:32.099189 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c85f7-f4bb-4a5a-b569-af94f3205c11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:32.101290 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c85f7-f4bb-4a5a-b569-af94f3205c11 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:32.245736 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 843/1672] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:03:31 2022] POST /identity/v3/users => generated 293 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:32.249698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adeaae1-3219-4064-8fb1-2ce08913cb59 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:32.251219 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-4adeaae1-3219-4064-8fb1-2ce08913cb59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:32.278886 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adeaae1-3219-4064-8fb1-2ce08913cb59 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:32.298455 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc065f-60ab-447d-abb7-16646bda10a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:32.299106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc065f-60ab-447d-abb7-16646bda10a0 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:32.299385 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc065f-60ab-447d-abb7-16646bda10a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:32.303665 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc065f-60ab-447d-abb7-16646bda10a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:32.304481 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fc065f-60ab-447d-abb7-16646bda10a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:32.306138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fc065f-60ab-447d-abb7-16646bda10a0 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:32.379648 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 830/1673] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:03:32.379911 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c85f7-f4bb-4a5a-b569-af94f3205c11 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:32.392947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c85f7-f4bb-4a5a-b569-af94f3205c11 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:32.395182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc065f-60ab-447d-abb7-16646bda10a0 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:32.400554 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-e5fc065f-60ab-447d-abb7-16646bda10a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:32.411879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7819c-1631-4cf3-be3d-bab9a44c00e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:32.415518 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7819c-1631-4cf3-be3d-bab9a44c00e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:32.415518 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7819c-1631-4cf3-be3d-bab9a44c00e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:32.418768 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7819c-1631-4cf3-be3d-bab9a44c00e1 admin admin] PATH_INFO: `/v3/projects/ad6bd7f10d714fd9a6697ec06fbc72a4/users/bda0864849c045199ee90cba6553d5ea/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:32.463210 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc065f-60ab-447d-abb7-16646bda10a0 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:32.481210 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc312533-e657-49e0-9471-21e52fa8c6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:32.481915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc312533-e657-49e0-9471-21e52fa8c6a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:32.482253 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc312533-e657-49e0-9471-21e52fa8c6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:32.482571 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc312533-e657-49e0-9471-21e52fa8c6a3 admin admin] PATH_INFO: `/v3/users/9497ff14b8974ea78de5c71775050e68` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:32.485114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc312533-e657-49e0-9471-21e52fa8c6a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9497ff14b8974ea78de5c71775050e68)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:32.491019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc312533-e657-49e0-9471-21e52fa8c6a3 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:32.523642 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:32.528324 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae7819c-1631-4cf3-be3d-bab9a44c00e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad6bd7f10d714fd9a6697ec06fbc72a4, user_id=bda0864849c045199ee90cba6553d5ea, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:32.531420 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae7819c-1631-4cf3-be3d-bab9a44c00e1 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:32.650823 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:32.659229 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 844/1674] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:32 2022] PUT /identity/v3/projects/ad6bd7f10d714fd9a6697ec06fbc72a4/users/bda0864849c045199ee90cba6553d5ea/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:32.660938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fb767-6958-46b6-822f-56eb7534fd16 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:32.662895 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fb767-6958-46b6-822f-56eb7534fd16 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:32.667137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fb767-6958-46b6-822f-56eb7534fd16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:32.769108 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-195fb767-6958-46b6-822f-56eb7534fd16 None None] MFA Rules not processed for user `d1c0bed0663c4afb8c59dd77be320905`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:32.776846 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-195fb767-6958-46b6-822f-56eb7534fd16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:32.840247 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-dc312533-e657-49e0-9471-21e52fa8c6a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9497ff14b8974ea78de5c71775050e68'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:32.882967 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 845/1675] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:32.888130 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195fb767-6958-46b6-822f-56eb7534fd16 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:32.953443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-dc312533-e657-49e0-9471-21e52fa8c6a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9497ff14b8974ea78de5c71775050e68'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:32.954452 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-dc312533-e657-49e0-9471-21e52fa8c6a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9497ff14b8974ea78de5c71775050e68'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:32.961420 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-dc312533-e657-49e0-9471-21e52fa8c6a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9497ff14b8974ea78de5c71775050e68'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:32.976511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-dc312533-e657-49e0-9471-21e52fa8c6a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9497ff14b8974ea78de5c71775050e68'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:33.003679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 831/1676] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:03:32 2022] DELETE /identity/v3/users/9497ff14b8974ea78de5c71775050e68 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:33.005294 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfd5bf-9ed6-495b-bdd3-dd907c63f8fc None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:33.005587 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfd5bf-9ed6-495b-bdd3-dd907c63f8fc None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:33.005857 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfd5bf-9ed6-495b-bdd3-dd907c63f8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:33.041082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-22dfd5bf-9ed6-495b-bdd3-dd907c63f8fc None None] MFA Rules not processed for user `d1c0bed0663c4afb8c59dd77be320905`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:33.043471 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-22dfd5bf-9ed6-495b-bdd3-dd907c63f8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:33.051879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195fb767-6958-46b6-822f-56eb7534fd16 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:33.175863 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 832/1677] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:33.177622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965adf61-51ea-44c8-bd62-8ba432e23d2a None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:33.177974 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965adf61-51ea-44c8-bd62-8ba432e23d2a None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:33.178501 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965adf61-51ea-44c8-bd62-8ba432e23d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:33.188955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93592a5d-eb77-4a2e-a328-82b84001c817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:33.190020 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93592a5d-eb77-4a2e-a328-82b84001c817 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:33.190576 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93592a5d-eb77-4a2e-a328-82b84001c817 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:33.191020 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93592a5d-eb77-4a2e-a328-82b84001c817 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:33.191822 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93592a5d-eb77-4a2e-a328-82b84001c817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:33.193877 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93592a5d-eb77-4a2e-a328-82b84001c817 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:33.210074 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 846/1678] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:33.211308 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93592a5d-eb77-4a2e-a328-82b84001c817 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:33.213230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-93592a5d-eb77-4a2e-a328-82b84001c817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:33.216316 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-965adf61-51ea-44c8-bd62-8ba432e23d2a None None] MFA Rules not processed for user `927bd33801e34715a78b58b268e28c4b`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:33.216898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-965adf61-51ea-44c8-bd62-8ba432e23d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:33.358849 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 833/1679] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:33 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:33.372545 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965adf61-51ea-44c8-bd62-8ba432e23d2a None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:33.372545 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-965adf61-51ea-44c8-bd62-8ba432e23d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:33.410294 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965adf61-51ea-44c8-bd62-8ba432e23d2a None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:33.422759 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5be134-8837-438d-9aa7-3ff18a7ed27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:33.426888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93592a5d-eb77-4a2e-a328-82b84001c817 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:33.427009 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5be134-8837-438d-9aa7-3ff18a7ed27d admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:33.427009 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5be134-8837-438d-9aa7-3ff18a7ed27d admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:33.427009 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5be134-8837-438d-9aa7-3ff18a7ed27d admin admin] PATH_INFO: `/v3/projects/ad6bd7f10d714fd9a6697ec06fbc72a4/users/bda0864849c045199ee90cba6553d5ea/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:33.435243 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5be134-8837-438d-9aa7-3ff18a7ed27d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad6bd7f10d714fd9a6697ec06fbc72a4, user_id=bda0864849c045199ee90cba6553d5ea, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:33.439666 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec8846-0525-4de8-b23a-3ed9a3a4ee53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:33.440919 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec8846-0525-4de8-b23a-3ed9a3a4ee53 service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:33.442064 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5be134-8837-438d-9aa7-3ff18a7ed27d admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:33.442172 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec8846-0525-4de8-b23a-3ed9a3a4ee53 service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:33.442340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec8846-0525-4de8-b23a-3ed9a3a4ee53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:33.449269 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-36ec8846-0525-4de8-b23a-3ed9a3a4ee53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:33.461885 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 834/1680] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:33 2022] PUT /identity/v3/projects/ad6bd7f10d714fd9a6697ec06fbc72a4/users/bda0864849c045199ee90cba6553d5ea/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:33.463184 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a27e7a-4bf1-4c50-841b-492237fef124 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:33.463287 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a27e7a-4bf1-4c50-841b-492237fef124 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:33.464234 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a27e7a-4bf1-4c50-841b-492237fef124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:33.481422 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec8846-0525-4de8-b23a-3ed9a3a4ee53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:33.483169 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec8846-0525-4de8-b23a-3ed9a3a4ee53 service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:33.504247 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 847/1681] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:33.505418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906b60d-4e73-4148-a63e-39bbd224fe61 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:33.505728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906b60d-4e73-4148-a63e-39bbd224fe61 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:33.505990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906b60d-4e73-4148-a63e-39bbd224fe61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:33.517417 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-29a27e7a-4bf1-4c50-841b-492237fef124 None None] MFA Rules not processed for user `927bd33801e34715a78b58b268e28c4b`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:33.518676 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-29a27e7a-4bf1-4c50-841b-492237fef124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:33.580392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-f906b60d-4e73-4148-a63e-39bbd224fe61 None None] MFA Rules not processed for user `bda0864849c045199ee90cba6553d5ea`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:33.581714 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-f906b60d-4e73-4148-a63e-39bbd224fe61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:33.710246 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 835/1682] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:33 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:33.721762 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a27e7a-4bf1-4c50-841b-492237fef124 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:33.723256 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-29a27e7a-4bf1-4c50-841b-492237fef124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:33.775295 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 848/1683] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:33.776770 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906b60d-4e73-4148-a63e-39bbd224fe61 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:33.778410 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-f906b60d-4e73-4148-a63e-39bbd224fe61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:33.889550 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a27e7a-4bf1-4c50-841b-492237fef124 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:33.893080 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a4175-4cea-4eaa-9371-fa08f28f8401 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:33.894029 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a4175-4cea-4eaa-9371-fa08f28f8401 service nova] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:33.894157 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a4175-4cea-4eaa-9371-fa08f28f8401 service nova] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:33.894533 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a4175-4cea-4eaa-9371-fa08f28f8401 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:33.912079 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a4175-4cea-4eaa-9371-fa08f28f8401 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:33.919516 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a4175-4cea-4eaa-9371-fa08f28f8401 service nova] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:33.952783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 836/1684] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:33 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:33.954032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a4175-4cea-4eaa-9371-fa08f28f8401 service nova] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:33.960898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-481a4175-4cea-4eaa-9371-fa08f28f8401 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:33.997700 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906b60d-4e73-4148-a63e-39bbd224fe61 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:34.014887 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a6239-d11c-4048-946c-7853c8768de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:34.015284 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a6239-d11c-4048-946c-7853c8768de9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:34.015567 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a6239-d11c-4048-946c-7853c8768de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:34.015841 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a6239-d11c-4048-946c-7853c8768de9 admin admin] PATH_INFO: `/v3/projects/f7a3570ed60040acb043931071a7db89` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:34.017432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a6239-d11c-4048-946c-7853c8768de9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7a3570ed60040acb043931071a7db89)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:34.019441 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a6239-d11c-4048-946c-7853c8768de9 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:34.150774 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a4175-4cea-4eaa-9371-fa08f28f8401 service nova] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:34.155267 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81603d9d-8530-4975-ac0c-7112ad49e339 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:34.155908 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81603d9d-8530-4975-ac0c-7112ad49e339 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:34.156189 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81603d9d-8530-4975-ac0c-7112ad49e339 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:34.156464 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81603d9d-8530-4975-ac0c-7112ad49e339 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:34.168462 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81603d9d-8530-4975-ac0c-7112ad49e339 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:34.170214 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81603d9d-8530-4975-ac0c-7112ad49e339 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:34.206478 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 837/1685] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:34.207729 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81603d9d-8530-4975-ac0c-7112ad49e339 service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:34.212754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-81603d9d-8530-4975-ac0c-7112ad49e339 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:34.248665 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81603d9d-8530-4975-ac0c-7112ad49e339 service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:34.277490 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-bd8a6239-d11c-4048-946c-7853c8768de9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7a3570ed60040acb043931071a7db89', 'request_id': 'req-bd8a6239-d11c-4048-946c-7853c8768de9'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:34.334383 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe3b5ba-2d93-4907-bf51-60e7ac376e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:34.335088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3b5ba-2d93-4907-bf51-60e7ac376e29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:34.335465 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3b5ba-2d93-4907-bf51-60e7ac376e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:34.335749 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3b5ba-2d93-4907-bf51-60e7ac376e29 admin admin] PATH_INFO: `/v3/projects/0b740784a59a48b6bfdf0e9643a05dc8` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:34.340462 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe3b5ba-2d93-4907-bf51-60e7ac376e29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b740784a59a48b6bfdf0e9643a05dc8)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:34.341513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: WARNING py.warnings [None req-fbe3b5ba-2d93-4907-bf51-60e7ac376e29 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:03:34.341513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: warnings.warn(msg) Aug 31 11:03:34.341513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: Aug 31 11:03:34.342784 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe3b5ba-2d93-4907-bf51-60e7ac376e29 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:34.436458 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-bd8a6239-d11c-4048-946c-7853c8768de9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7a3570ed60040acb043931071a7db89', 'request_id': 'req-bd8a6239-d11c-4048-946c-7853c8768de9'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:34.465037 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-bd8a6239-d11c-4048-946c-7853c8768de9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7a3570ed60040acb043931071a7db89', 'request_id': 'req-bd8a6239-d11c-4048-946c-7853c8768de9'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:34.535775 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:34.541563 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-bd8a6239-d11c-4048-946c-7853c8768de9 admin admin] The token cache is being invalidate because project f7a3570ed60040acb043931071a7db89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:03:34.545654 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-bd8a6239-d11c-4048-946c-7853c8768de9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:34.545654 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-bd8a6239-d11c-4048-946c-7853c8768de9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:34.554328 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 849/1686] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:03:33 2022] DELETE /identity/v3/projects/f7a3570ed60040acb043931071a7db89 => generated 0 bytes in 778 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:34.558645 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a6239-d11c-4048-946c-7853c8768de9 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:34.562700 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-bd8a6239-d11c-4048-946c-7853c8768de9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:34.590120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a6239-d11c-4048-946c-7853c8768de9 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:34.646585 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-fbe3b5ba-2d93-4907-bf51-60e7ac376e29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b740784a59a48b6bfdf0e9643a05dc8', 'request_id': 'req-fbe3b5ba-2d93-4907-bf51-60e7ac376e29'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:34.675333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-fbe3b5ba-2d93-4907-bf51-60e7ac376e29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b740784a59a48b6bfdf0e9643a05dc8', 'request_id': 'req-fbe3b5ba-2d93-4907-bf51-60e7ac376e29'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:34.706033 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-fbe3b5ba-2d93-4907-bf51-60e7ac376e29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b740784a59a48b6bfdf0e9643a05dc8', 'request_id': 'req-fbe3b5ba-2d93-4907-bf51-60e7ac376e29'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:34.708515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab935c-130f-4918-bbc2-d4cca1f62946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:34.716739 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab935c-130f-4918-bbc2-d4cca1f62946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:34.717340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab935c-130f-4918-bbc2-d4cca1f62946 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:34.719056 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab935c-130f-4918-bbc2-d4cca1f62946 admin admin] PATH_INFO: `/v3/users/d0f259a199404c70b66531c38361685c` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:34.726982 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab935c-130f-4918-bbc2-d4cca1f62946 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0f259a199404c70b66531c38361685c)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:34.733078 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab935c-130f-4918-bbc2-d4cca1f62946 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:34.826739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-fbe3b5ba-2d93-4907-bf51-60e7ac376e29 admin admin] The token cache is being invalidate because project 0b740784a59a48b6bfdf0e9643a05dc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:03:34.828642 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-fbe3b5ba-2d93-4907-bf51-60e7ac376e29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:34.828642 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-fbe3b5ba-2d93-4907-bf51-60e7ac376e29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:34.834435 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 838/1687] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:03:34 2022] DELETE /identity/v3/projects/0b740784a59a48b6bfdf0e9643a05dc8 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:34.835758 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c313470-5422-44e2-b7ad-7703a9c6948f None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:34.836045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c313470-5422-44e2-b7ad-7703a9c6948f None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:34.836315 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c313470-5422-44e2-b7ad-7703a9c6948f None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:34.837793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 839/1688] 10.222.0.16 () {54 vars in 912 bytes} [Wed Aug 31 11:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:03:34.838334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c313470-5422-44e2-b7ad-7703a9c6948f None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:34.841107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-5c313470-5422-44e2-b7ad-7703a9c6948f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:34.868725 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c313470-5422-44e2-b7ad-7703a9c6948f None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:34.876977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960ae55-d7c8-474b-a578-1bec2b45955c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:34.877082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960ae55-d7c8-474b-a578-1bec2b45955c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:34.878049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960ae55-d7c8-474b-a578-1bec2b45955c admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:34.878539 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960ae55-d7c8-474b-a578-1bec2b45955c admin admin] PATH_INFO: `/v3/users/e5e5c0bd424943789b1f18e26344c8ab` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:34.883963 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7960ae55-d7c8-474b-a578-1bec2b45955c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5e5c0bd424943789b1f18e26344c8ab)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:34.887083 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7960ae55-d7c8-474b-a578-1bec2b45955c admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:35.024009 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c7ab935c-130f-4918-bbc2-d4cca1f62946 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0f259a199404c70b66531c38361685c'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:35.043738 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c7ab935c-130f-4918-bbc2-d4cca1f62946 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0f259a199404c70b66531c38361685c'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:35.051702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c7ab935c-130f-4918-bbc2-d4cca1f62946 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0f259a199404c70b66531c38361685c'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:35.141308 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c7ab935c-130f-4918-bbc2-d4cca1f62946 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0f259a199404c70b66531c38361685c'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:35.141308 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c7ab935c-130f-4918-bbc2-d4cca1f62946 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0f259a199404c70b66531c38361685c'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:35.154498 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 850/1689] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:03:34 2022] DELETE /identity/v3/users/d0f259a199404c70b66531c38361685c => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:35.156332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d2927-e1ca-4691-9a16-b1720d99d38b None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:35.160423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d2927-e1ca-4691-9a16-b1720d99d38b None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:35.161479 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d2927-e1ca-4691-9a16-b1720d99d38b None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:35.163091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 851/1690] 10.222.0.16 () {54 vars in 912 bytes} [Wed Aug 31 11:03:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:03:35.174259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cc801-7e17-44ec-9de4-99e7d92b99bb None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:35.178414 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cc801-7e17-44ec-9de4-99e7d92b99bb None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:35.178755 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cc801-7e17-44ec-9de4-99e7d92b99bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:35.186930 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-7960ae55-d7c8-474b-a578-1bec2b45955c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5e5c0bd424943789b1f18e26344c8ab'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:35.233699 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-518cc801-7e17-44ec-9de4-99e7d92b99bb None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:35.235472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-518cc801-7e17-44ec-9de4-99e7d92b99bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:35.294721 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-7960ae55-d7c8-474b-a578-1bec2b45955c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5e5c0bd424943789b1f18e26344c8ab'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:35.295374 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-7960ae55-d7c8-474b-a578-1bec2b45955c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5e5c0bd424943789b1f18e26344c8ab'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:35.296628 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-7960ae55-d7c8-474b-a578-1bec2b45955c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5e5c0bd424943789b1f18e26344c8ab'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:35.319563 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-7960ae55-d7c8-474b-a578-1bec2b45955c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5e5c0bd424943789b1f18e26344c8ab'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:35.356860 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 840/1691] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:03:34 2022] DELETE /identity/v3/users/e5e5c0bd424943789b1f18e26344c8ab => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:35.371646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d0014-588f-402e-8772-677607cb0e7b None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:35.371931 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d0014-588f-402e-8772-677607cb0e7b None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:35.373409 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d0014-588f-402e-8772-677607cb0e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:35.445596 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 852/1692] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:35.445596 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e869dc7-175d-4cc9-908d-e0db0543906b None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:35.445596 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e869dc7-175d-4cc9-908d-e0db0543906b None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:35.445596 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e869dc7-175d-4cc9-908d-e0db0543906b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:35.472308 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-d72d0014-588f-402e-8772-677607cb0e7b None None] MFA Rules not processed for user `1a00b7f45b2e47b8a01048c36d00f742`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:35.473557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-d72d0014-588f-402e-8772-677607cb0e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:35.484978 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-9e869dc7-175d-4cc9-908d-e0db0543906b None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:35.486236 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-9e869dc7-175d-4cc9-908d-e0db0543906b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:35.584732 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 841/1693] 10.222.0.16 () {58 vars in 1009 bytes} [Wed Aug 31 11:03:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:35.586020 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34a89b-69f1-400a-839c-465f304dd603 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:35.586316 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34a89b-69f1-400a-839c-465f304dd603 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:35.586591 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34a89b-69f1-400a-839c-465f304dd603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:35.613133 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:35.626659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 853/1694] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:35.628856 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370803f-952c-4b9f-96e5-a7951efffd1b None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:35.629141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370803f-952c-4b9f-96e5-a7951efffd1b None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:35.629413 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370803f-952c-4b9f-96e5-a7951efffd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:35.656292 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-9370803f-952c-4b9f-96e5-a7951efffd1b None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:35.657541 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-9370803f-952c-4b9f-96e5-a7951efffd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:35.666806 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 854/1695] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:35.670815 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370803f-952c-4b9f-96e5-a7951efffd1b None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:35.684957 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-5c34a89b-69f1-400a-839c-465f304dd603 None None] MFA Rules not processed for user `1a00b7f45b2e47b8a01048c36d00f742`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:35.686490 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-5c34a89b-69f1-400a-839c-465f304dd603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:35.689960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370803f-952c-4b9f-96e5-a7951efffd1b None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:35.695987 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa2bf74-e8f9-4898-b209-62581966efc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:35.696877 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2bf74-e8f9-4898-b209-62581966efc0 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:35.697348 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2bf74-e8f9-4898-b209-62581966efc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:35.698434 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2bf74-e8f9-4898-b209-62581966efc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:35.699548 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa2bf74-e8f9-4898-b209-62581966efc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:03:35.700801 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa2bf74-e8f9-4898-b209-62581966efc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:35.705354 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa2bf74-e8f9-4898-b209-62581966efc0 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:35.724451 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 842/1696] 10.222.0.16 () {58 vars in 1009 bytes} [Wed Aug 31 11:03:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:35.727863 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c34a89b-69f1-400a-839c-465f304dd603 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:35.730268 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-5c34a89b-69f1-400a-839c-465f304dd603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:35.777240 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c34a89b-69f1-400a-839c-465f304dd603 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:35.782089 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 855/1697] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:03:35.784565 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa2bf74-e8f9-4898-b209-62581966efc0 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:35.785809 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edffc6f-af68-46e6-9217-80178a83b00e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:35.786494 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edffc6f-af68-46e6-9217-80178a83b00e service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:35.786777 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edffc6f-af68-46e6-9217-80178a83b00e service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:35.787049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edffc6f-af68-46e6-9217-80178a83b00e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:35.807081 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edffc6f-af68-46e6-9217-80178a83b00e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:35.812463 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edffc6f-af68-46e6-9217-80178a83b00e service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:35.819643 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa2bf74-e8f9-4898-b209-62581966efc0 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:35.832739 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f834dac-5ce1-4b27-abbf-44936bbbd580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:35.833635 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f834dac-5ce1-4b27-abbf-44936bbbd580 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:35.836026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 843/1698] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:35.837145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edffc6f-af68-46e6-9217-80178a83b00e service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:35.847503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f834dac-5ce1-4b27-abbf-44936bbbd580 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:35.850011 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f834dac-5ce1-4b27-abbf-44936bbbd580 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:35.850886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f834dac-5ce1-4b27-abbf-44936bbbd580 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:35.852945 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f834dac-5ce1-4b27-abbf-44936bbbd580 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:35.856520 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edffc6f-af68-46e6-9217-80178a83b00e service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:35.867088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917a9f9-331a-4607-b8db-0a5e7d445f47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:35.875240 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917a9f9-331a-4607-b8db-0a5e7d445f47 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:35.875750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917a9f9-331a-4607-b8db-0a5e7d445f47 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:35.876193 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917a9f9-331a-4607-b8db-0a5e7d445f47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:35.890344 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917a9f9-331a-4607-b8db-0a5e7d445f47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:35.892505 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917a9f9-331a-4607-b8db-0a5e7d445f47 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:35.913685 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 844/1699] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:35.933339 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917a9f9-331a-4607-b8db-0a5e7d445f47 service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:35.935424 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-3917a9f9-331a-4607-b8db-0a5e7d445f47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:35.953300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917a9f9-331a-4607-b8db-0a5e7d445f47 service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:35.975927 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 856/1700] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:03:35 2022] POST /identity/v3/projects => generated 377 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:35.981596 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f834dac-5ce1-4b27-abbf-44936bbbd580 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:36.138666 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389cbf64-f17f-4f69-84b2-c292125fc8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:36.140897 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cbf64-f17f-4f69-84b2-c292125fc8b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:36.140897 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cbf64-f17f-4f69-84b2-c292125fc8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:36.140897 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cbf64-f17f-4f69-84b2-c292125fc8b9 admin admin] PATH_INFO: `/v3/users/9b8a926f129b44749ce1d95f05af5357` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:36.147547 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f834dac-5ce1-4b27-abbf-44936bbbd580 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:36.149399 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389cbf64-f17f-4f69-84b2-c292125fc8b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b8a926f129b44749ce1d95f05af5357)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:36.152423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389cbf64-f17f-4f69-84b2-c292125fc8b9 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:36.172585 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d936c70-e972-4ea3-b064-d97289442ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:36.175312 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d936c70-e972-4ea3-b064-d97289442ad0 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:36.175312 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d936c70-e972-4ea3-b064-d97289442ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:36.175312 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d936c70-e972-4ea3-b064-d97289442ad0 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:36.177532 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d936c70-e972-4ea3-b064-d97289442ad0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:36.185257 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d936c70-e972-4ea3-b064-d97289442ad0 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:36.312989 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 857/1701] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:03:35 2022] POST /identity/v3/users => generated 289 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:36.312989 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d936c70-e972-4ea3-b064-d97289442ad0 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:36.314569 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-0d936c70-e972-4ea3-b064-d97289442ad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:36.418209 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-389cbf64-f17f-4f69-84b2-c292125fc8b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b8a926f129b44749ce1d95f05af5357'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:36.451878 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d936c70-e972-4ea3-b064-d97289442ad0 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:36.456178 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565108d-0fcf-4509-9d22-2f9cba51108c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:36.456879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565108d-0fcf-4509-9d22-2f9cba51108c service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:36.457241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565108d-0fcf-4509-9d22-2f9cba51108c service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:36.458871 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565108d-0fcf-4509-9d22-2f9cba51108c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:36.470613 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6565108d-0fcf-4509-9d22-2f9cba51108c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:36.472161 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6565108d-0fcf-4509-9d22-2f9cba51108c service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:36.482779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 858/1702] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:36.483601 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565108d-0fcf-4509-9d22-2f9cba51108c service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:36.485151 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-6565108d-0fcf-4509-9d22-2f9cba51108c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:36.501930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565108d-0fcf-4509-9d22-2f9cba51108c service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:36.513825 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99bdb02-8814-4449-9feb-62433c8bf7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:36.514476 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99bdb02-8814-4449-9feb-62433c8bf7e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:36.514892 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99bdb02-8814-4449-9feb-62433c8bf7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:36.515177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99bdb02-8814-4449-9feb-62433c8bf7e5 admin admin] PATH_INFO: `/v3/projects/4902aae602384b9196f679ce77cfb4b8` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:36.516544 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99bdb02-8814-4449-9feb-62433c8bf7e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4902aae602384b9196f679ce77cfb4b8)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:36.517194 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-389cbf64-f17f-4f69-84b2-c292125fc8b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b8a926f129b44749ce1d95f05af5357'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:36.517830 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-389cbf64-f17f-4f69-84b2-c292125fc8b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b8a926f129b44749ce1d95f05af5357'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:36.518553 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-389cbf64-f17f-4f69-84b2-c292125fc8b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b8a926f129b44749ce1d95f05af5357'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:36.525346 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99bdb02-8814-4449-9feb-62433c8bf7e5 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:36.543948 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-389cbf64-f17f-4f69-84b2-c292125fc8b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b8a926f129b44749ce1d95f05af5357'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:36.578314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 845/1703] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:03:35 2022] DELETE /identity/v3/users/9b8a926f129b44749ce1d95f05af5357 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:36.580624 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389cbf64-f17f-4f69-84b2-c292125fc8b9 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:36.586155 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-389cbf64-f17f-4f69-84b2-c292125fc8b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:36.747450 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389cbf64-f17f-4f69-84b2-c292125fc8b9 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:36.755964 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e99bdb02-8814-4449-9feb-62433c8bf7e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4902aae602384b9196f679ce77cfb4b8', 'request_id': 'req-e99bdb02-8814-4449-9feb-62433c8bf7e5'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:36.827291 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e99bdb02-8814-4449-9feb-62433c8bf7e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4902aae602384b9196f679ce77cfb4b8', 'request_id': 'req-e99bdb02-8814-4449-9feb-62433c8bf7e5'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:36.839903 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e99bdb02-8814-4449-9feb-62433c8bf7e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4902aae602384b9196f679ce77cfb4b8', 'request_id': 'req-e99bdb02-8814-4449-9feb-62433c8bf7e5'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:36.874843 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e99bdb02-8814-4449-9feb-62433c8bf7e5 admin admin] The token cache is being invalidate because project 4902aae602384b9196f679ce77cfb4b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:03:36.875144 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e99bdb02-8814-4449-9feb-62433c8bf7e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:36.876002 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e99bdb02-8814-4449-9feb-62433c8bf7e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:36.877776 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 859/1704] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:03:36 2022] DELETE /identity/v3/projects/4902aae602384b9196f679ce77cfb4b8 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:36.879250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86ba11-fcfb-4619-b7cf-f3c155e0636d None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:36.879529 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86ba11-fcfb-4619-b7cf-f3c155e0636d None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:36.879805 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86ba11-fcfb-4619-b7cf-f3c155e0636d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:36.883494 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e7c02-fca5-414d-9520-bc5d7f405860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:36.884173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e7c02-fca5-414d-9520-bc5d7f405860 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:36.884449 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e7c02-fca5-414d-9520-bc5d7f405860 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:36.884712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e7c02-fca5-414d-9520-bc5d7f405860 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:36.886113 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6e7c02-fca5-414d-9520-bc5d7f405860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:36.888041 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6e7c02-fca5-414d-9520-bc5d7f405860 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:36.902375 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 846/1705] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:36.903752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38438b-1650-4fe9-8c03-5ce206644937 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:36.904037 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38438b-1650-4fe9-8c03-5ce206644937 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:36.904203 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-2d86ba11-fcfb-4619-b7cf-f3c155e0636d None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:36.904308 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38438b-1650-4fe9-8c03-5ce206644937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:36.905837 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2d86ba11-fcfb-4619-b7cf-f3c155e0636d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:36.919622 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 860/1706] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:36.920576 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86ba11-fcfb-4619-b7cf-f3c155e0636d None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:36.921991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2d86ba11-fcfb-4619-b7cf-f3c155e0636d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:36.936385 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-9d38438b-1650-4fe9-8c03-5ce206644937 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:36.937011 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86ba11-fcfb-4619-b7cf-f3c155e0636d None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:36.937755 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-9d38438b-1650-4fe9-8c03-5ce206644937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:36.947307 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c157918-f5fe-41e0-ac1d-05b9455997a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:36.947980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c157918-f5fe-41e0-ac1d-05b9455997a7 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:36.948299 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c157918-f5fe-41e0-ac1d-05b9455997a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:36.948576 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c157918-f5fe-41e0-ac1d-05b9455997a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:36.949300 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c157918-f5fe-41e0-ac1d-05b9455997a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:36.951991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c157918-f5fe-41e0-ac1d-05b9455997a7 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:36.974750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 847/1707] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:36.975600 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d38438b-1650-4fe9-8c03-5ce206644937 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:36.982078 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 861/1708] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:36.982361 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-9d38438b-1650-4fe9-8c03-5ce206644937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:36.982879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c157918-f5fe-41e0-ac1d-05b9455997a7 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:36.988620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-7c157918-f5fe-41e0-ac1d-05b9455997a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:37.017588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c157918-f5fe-41e0-ac1d-05b9455997a7 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:37.026622 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057f67a-6a9f-4353-a817-60d5fa34cc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:37.029054 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057f67a-6a9f-4353-a817-60d5fa34cc16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:37.029349 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057f67a-6a9f-4353-a817-60d5fa34cc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:37.030848 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057f67a-6a9f-4353-a817-60d5fa34cc16 admin admin] PATH_INFO: `/v3/projects/17643718ef274305a1c33bf09bedef88` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:37.033498 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9057f67a-6a9f-4353-a817-60d5fa34cc16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17643718ef274305a1c33bf09bedef88)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:37.046848 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9057f67a-6a9f-4353-a817-60d5fa34cc16 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:37.118000 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d38438b-1650-4fe9-8c03-5ce206644937 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:37.128121 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503b2de-1d86-4db8-a077-d4f2c504709e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:37.128766 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503b2de-1d86-4db8-a077-d4f2c504709e service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:37.129041 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503b2de-1d86-4db8-a077-d4f2c504709e service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:37.129307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503b2de-1d86-4db8-a077-d4f2c504709e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:37.153135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503b2de-1d86-4db8-a077-d4f2c504709e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:37.156173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503b2de-1d86-4db8-a077-d4f2c504709e service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:37.181408 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 848/1709] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:37.185634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc088905-a8e3-4de0-8154-2d382d7f3a54 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:37.186421 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc088905-a8e3-4de0-8154-2d382d7f3a54 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:37.187606 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc088905-a8e3-4de0-8154-2d382d7f3a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:37.233229 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-9057f67a-6a9f-4353-a817-60d5fa34cc16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17643718ef274305a1c33bf09bedef88', 'request_id': 'req-9057f67a-6a9f-4353-a817-60d5fa34cc16'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:37.264163 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-dc088905-a8e3-4de0-8154-2d382d7f3a54 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:37.276212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-dc088905-a8e3-4de0-8154-2d382d7f3a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:37.366662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 849/1710] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:37.366662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc088905-a8e3-4de0-8154-2d382d7f3a54 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:37.380602 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc088905-a8e3-4de0-8154-2d382d7f3a54 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:37.387265 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca6054-119b-43e6-a1b3-e43954f1327b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:37.388025 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-9057f67a-6a9f-4353-a817-60d5fa34cc16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17643718ef274305a1c33bf09bedef88', 'request_id': 'req-9057f67a-6a9f-4353-a817-60d5fa34cc16'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:37.388162 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca6054-119b-43e6-a1b3-e43954f1327b admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:37.388405 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca6054-119b-43e6-a1b3-e43954f1327b admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:37.388684 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca6054-119b-43e6-a1b3-e43954f1327b admin admin] PATH_INFO: `/v3/projects/125145a02a6b4b32959536cf97a3b0df/users/044355b41ef645b386d54e1955c0417c/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:37.398476 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-9057f67a-6a9f-4353-a817-60d5fa34cc16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17643718ef274305a1c33bf09bedef88', 'request_id': 'req-9057f67a-6a9f-4353-a817-60d5fa34cc16'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:37.407083 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ca6054-119b-43e6-a1b3-e43954f1327b admin admin] RBAC: Authorizing `identity:create_grant(project_id=125145a02a6b4b32959536cf97a3b0df, user_id=044355b41ef645b386d54e1955c0417c, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:37.410113 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ca6054-119b-43e6-a1b3-e43954f1327b admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:37.421773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-9057f67a-6a9f-4353-a817-60d5fa34cc16 admin admin] The token cache is being invalidate because project 17643718ef274305a1c33bf09bedef88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:03:37.422356 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-9057f67a-6a9f-4353-a817-60d5fa34cc16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:37.423561 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-9057f67a-6a9f-4353-a817-60d5fa34cc16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:37.425748 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 862/1711] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:03:36 2022] DELETE /identity/v3/projects/17643718ef274305a1c33bf09bedef88 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:37.426426 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:37.427084 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057f67a-6a9f-4353-a817-60d5fa34cc16 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:37.428958 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-9057f67a-6a9f-4353-a817-60d5fa34cc16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:37.443329 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057f67a-6a9f-4353-a817-60d5fa34cc16 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:37.449656 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9bf1e7-143a-4ef5-9d6c-0e0ca15f17f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:37.450731 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bf1e7-143a-4ef5-9d6c-0e0ca15f17f4 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:37.451083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bf1e7-143a-4ef5-9d6c-0e0ca15f17f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:37.451427 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bf1e7-143a-4ef5-9d6c-0e0ca15f17f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:37.452528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9bf1e7-143a-4ef5-9d6c-0e0ca15f17f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:03:37.452845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9bf1e7-143a-4ef5-9d6c-0e0ca15f17f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:37.454567 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9bf1e7-143a-4ef5-9d6c-0e0ca15f17f4 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:37.488623 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 863/1712] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:03:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:03:37.489800 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9bf1e7-143a-4ef5-9d6c-0e0ca15f17f4 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:37.499521 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-7f9bf1e7-143a-4ef5-9d6c-0e0ca15f17f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:37.577626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:37.604506 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 850/1713] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:37 2022] PUT /identity/v3/projects/125145a02a6b4b32959536cf97a3b0df/users/044355b41ef645b386d54e1955c0417c/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:37.617638 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9bf1e7-143a-4ef5-9d6c-0e0ca15f17f4 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:37.621959 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33706edf-5c18-42c5-b864-7ee97346d6e5 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:37.623095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33706edf-5c18-42c5-b864-7ee97346d6e5 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:37.623567 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33706edf-5c18-42c5-b864-7ee97346d6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:37.648890 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-33706edf-5c18-42c5-b864-7ee97346d6e5 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:37.650663 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-33706edf-5c18-42c5-b864-7ee97346d6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:37.778640 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3742c902-dd32-4c16-a973-5a3050f4a241 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3db46a4b4e74cc0bb2a7bd6f0e99105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:37.779634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3742c902-dd32-4c16-a973-5a3050f4a241 service placement] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:37.779987 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3742c902-dd32-4c16-a973-5a3050f4a241 service placement] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:37.780468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3742c902-dd32-4c16-a973-5a3050f4a241 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:37.783367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-3742c902-dd32-4c16-a973-5a3050f4a241 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:37.785852 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 851/1714] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:37.787030 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33706edf-5c18-42c5-b864-7ee97346d6e5 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:37.804940 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33706edf-5c18-42c5-b864-7ee97346d6e5 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:37.814558 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cddbd2-6c6e-4487-8552-ea2b7ac614d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:37.814558 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cddbd2-6c6e-4487-8552-ea2b7ac614d5 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:37.819716 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3742c902-dd32-4c16-a973-5a3050f4a241 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:37.821414 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3742c902-dd32-4c16-a973-5a3050f4a241 service placement] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:37.824273 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cddbd2-6c6e-4487-8552-ea2b7ac614d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:37.824594 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cddbd2-6c6e-4487-8552-ea2b7ac614d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:37.825310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cddbd2-6c6e-4487-8552-ea2b7ac614d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:37.827301 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cddbd2-6c6e-4487-8552-ea2b7ac614d5 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:37.851022 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 864/1715] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:37.859771 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255cead-9027-4e3e-a00c-4502a69c0025 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:37.859771 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255cead-9027-4e3e-a00c-4502a69c0025 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:37.859771 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255cead-9027-4e3e-a00c-4502a69c0025 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:37.859771 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 865/1716] 10.222.0.16 () {54 vars in 961 bytes} [Wed Aug 31 11:03:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:03:37.859771 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b3d53-a55b-4a9c-930a-f384f448d487 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:37.859771 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b3d53-a55b-4a9c-930a-f384f448d487 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:37.860219 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b3d53-a55b-4a9c-930a-f384f448d487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:37.904736 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-325b3d53-a55b-4a9c-930a-f384f448d487 None None] MFA Rules not processed for user `927bd33801e34715a78b58b268e28c4b`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:37.906652 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-325b3d53-a55b-4a9c-930a-f384f448d487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:37.981486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 852/1717] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:03:37 2022] POST /identity/v3/projects => generated 363 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:37.982953 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4144ee7-aa37-40a2-aae0-0971b98a4bf9 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:37.983266 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4144ee7-aa37-40a2-aae0-0971b98a4bf9 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:37.983541 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4144ee7-aa37-40a2-aae0-0971b98a4bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:38.020904 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-c4144ee7-aa37-40a2-aae0-0971b98a4bf9 None None] MFA Rules not processed for user `044355b41ef645b386d54e1955c0417c`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:38.022988 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-c4144ee7-aa37-40a2-aae0-0971b98a4bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:38.030024 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 866/1718] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:38.031278 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfea59d-ee15-48a4-99af-bbf10793da14 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:38.031564 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfea59d-ee15-48a4-99af-bbf10793da14 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:38.031839 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfea59d-ee15-48a4-99af-bbf10793da14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:38.057507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-ddfea59d-ee15-48a4-99af-bbf10793da14 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:38.059907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-ddfea59d-ee15-48a4-99af-bbf10793da14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:38.203351 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 853/1719] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:38.205105 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b947bde-7067-460b-b995-5db04dde940a None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:38.205527 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b947bde-7067-460b-b995-5db04dde940a None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:38.205942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b947bde-7067-460b-b995-5db04dde940a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:38.231867 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 867/1720] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:38.232898 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfea59d-ee15-48a4-99af-bbf10793da14 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:38.240356 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-1b947bde-7067-460b-b995-5db04dde940a None None] MFA Rules not processed for user `f3db46a4b4e74cc0bb2a7bd6f0e99105`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:38.241927 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-1b947bde-7067-460b-b995-5db04dde940a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:38.249278 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfea59d-ee15-48a4-99af-bbf10793da14 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:38.255178 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a34ee-3725-4d58-b352-0652af2e1636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:38.257146 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a34ee-3725-4d58-b352-0652af2e1636 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:38.257146 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a34ee-3725-4d58-b352-0652af2e1636 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:38.257146 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a34ee-3725-4d58-b352-0652af2e1636 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:38.257602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a34ee-3725-4d58-b352-0652af2e1636 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:38.259915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a34ee-3725-4d58-b352-0652af2e1636 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:38.355344 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 854/1721] 10.222.0.16 () {58 vars in 1058 bytes} [Wed Aug 31 11:03:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:38.359888 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b947bde-7067-460b-b995-5db04dde940a None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:38.359888 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-1b947bde-7067-460b-b995-5db04dde940a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:38.379133 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b947bde-7067-460b-b995-5db04dde940a None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:38.384782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4c260-c483-400c-9c69-4856adb7caae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:38.385477 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4c260-c483-400c-9c69-4856adb7caae admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:38.385794 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4c260-c483-400c-9c69-4856adb7caae admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:38.386139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4c260-c483-400c-9c69-4856adb7caae admin admin] PATH_INFO: `/v3/projects/c661575302c34003a49a4aa85548563e` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:38.387502 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e4c260-c483-400c-9c69-4856adb7caae admin admin] RBAC: Authorizing `identity:delete_project(project_id=c661575302c34003a49a4aa85548563e)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:38.389483 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e4c260-c483-400c-9c69-4856adb7caae admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:38.421980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 868/1722] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:03:38 2022] POST /identity/v3/users => generated 282 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:38.424842 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a34ee-3725-4d58-b352-0652af2e1636 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:38.428361 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-781a34ee-3725-4d58-b352-0652af2e1636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:38.448491 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a34ee-3725-4d58-b352-0652af2e1636 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:38.456091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ca3dc-368d-409d-9f23-3a7de6badc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:38.456855 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ca3dc-368d-409d-9f23-3a7de6badc28 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:38.457191 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ca3dc-368d-409d-9f23-3a7de6badc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:38.457474 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ca3dc-368d-409d-9f23-3a7de6badc28 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:38.458739 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831ca3dc-368d-409d-9f23-3a7de6badc28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:38.461744 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831ca3dc-368d-409d-9f23-3a7de6badc28 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:38.526438 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-43e4c260-c483-400c-9c69-4856adb7caae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c661575302c34003a49a4aa85548563e', 'request_id': 'req-43e4c260-c483-400c-9c69-4856adb7caae'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:38.547419 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 869/1723] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:03:38 2022] POST /identity/v3/projects => generated 377 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:38.548672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ca3dc-368d-409d-9f23-3a7de6badc28 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:38.555949 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-43e4c260-c483-400c-9c69-4856adb7caae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c661575302c34003a49a4aa85548563e', 'request_id': 'req-43e4c260-c483-400c-9c69-4856adb7caae'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:38.575715 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-43e4c260-c483-400c-9c69-4856adb7caae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c661575302c34003a49a4aa85548563e', 'request_id': 'req-43e4c260-c483-400c-9c69-4856adb7caae'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:38.708997 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ca3dc-368d-409d-9f23-3a7de6badc28 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:38.719995 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de43a5-ece2-41bc-a605-f2df5a0e288c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:38.720179 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de43a5-ece2-41bc-a605-f2df5a0e288c admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:38.720346 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de43a5-ece2-41bc-a605-f2df5a0e288c admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:38.720588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de43a5-ece2-41bc-a605-f2df5a0e288c admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:38.721259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de43a5-ece2-41bc-a605-f2df5a0e288c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:03:38.721617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de43a5-ece2-41bc-a605-f2df5a0e288c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:38.726311 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de43a5-ece2-41bc-a605-f2df5a0e288c admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:38.733260 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-43e4c260-c483-400c-9c69-4856adb7caae admin admin] The token cache is being invalidate because project c661575302c34003a49a4aa85548563e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:03:38.737504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-43e4c260-c483-400c-9c69-4856adb7caae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:38.737504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-43e4c260-c483-400c-9c69-4856adb7caae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:38.739141 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 855/1724] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:03:38 2022] DELETE /identity/v3/projects/c661575302c34003a49a4aa85548563e => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:38.740385 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410dbc0-5527-4b52-98bd-75552676e137 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:38.740659 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410dbc0-5527-4b52-98bd-75552676e137 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:38.740934 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410dbc0-5527-4b52-98bd-75552676e137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:38.775135 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 870/1725] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:03:38.776456 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de43a5-ece2-41bc-a605-f2df5a0e288c admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:38.779121 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-b0de43a5-ece2-41bc-a605-f2df5a0e288c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:38.782688 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-f410dbc0-5527-4b52-98bd-75552676e137 None None] MFA Rules not processed for user `d5d0010f584249b4a22fd31634c9fd8b`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:38.783973 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-f410dbc0-5527-4b52-98bd-75552676e137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:38.945487 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:38.949888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de43a5-ece2-41bc-a605-f2df5a0e288c admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:38.962874 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 856/1726] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:38.964369 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410dbc0-5527-4b52-98bd-75552676e137 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:38.970195 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91bdb0-f810-4bc5-9bd5-18d613be4dea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3db46a4b4e74cc0bb2a7bd6f0e99105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:38.970859 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91bdb0-f810-4bc5-9bd5-18d613be4dea service placement] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:38.971199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91bdb0-f810-4bc5-9bd5-18d613be4dea service placement] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:38.971400 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91bdb0-f810-4bc5-9bd5-18d613be4dea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:38.976856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-f410dbc0-5527-4b52-98bd-75552676e137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:38.978857 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-0b91bdb0-f810-4bc5-9bd5-18d613be4dea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:39.020157 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b91bdb0-f810-4bc5-9bd5-18d613be4dea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:39.020157 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b91bdb0-f810-4bc5-9bd5-18d613be4dea service placement] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:39.038731 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410dbc0-5527-4b52-98bd-75552676e137 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:39.056956 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 871/1727] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:39.056956 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91bdb0-f810-4bc5-9bd5-18d613be4dea service placement] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:39.056956 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-0b91bdb0-f810-4bc5-9bd5-18d613be4dea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:39.071444 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952dba35-d981-47e6-aebb-3e2f793c91f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:39.072078 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952dba35-d981-47e6-aebb-3e2f793c91f4 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:39.072352 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952dba35-d981-47e6-aebb-3e2f793c91f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:39.072758 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952dba35-d981-47e6-aebb-3e2f793c91f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:39.073355 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952dba35-d981-47e6-aebb-3e2f793c91f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:39.079429 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91bdb0-f810-4bc5-9bd5-18d613be4dea service placement] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:39.082954 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:39.087681 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952dba35-d981-47e6-aebb-3e2f793c91f4 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:39.094432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0f921-55f1-4c09-92ea-4a1a9555dbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:39.094432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0f921-55f1-4c09-92ea-4a1a9555dbaf admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:39.097679 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0f921-55f1-4c09-92ea-4a1a9555dbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:39.098207 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0f921-55f1-4c09-92ea-4a1a9555dbaf admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:39.105771 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0f921-55f1-4c09-92ea-4a1a9555dbaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:39.107780 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0f921-55f1-4c09-92ea-4a1a9555dbaf admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:39.148665 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 857/1728] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:39.149668 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952dba35-d981-47e6-aebb-3e2f793c91f4 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:39.152431 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-952dba35-d981-47e6-aebb-3e2f793c91f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:39.184211 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952dba35-d981-47e6-aebb-3e2f793c91f4 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:39.189608 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d285cd1-9ae8-453a-b2dc-bb092bc5f094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:39.190510 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d285cd1-9ae8-453a-b2dc-bb092bc5f094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:39.190980 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d285cd1-9ae8-453a-b2dc-bb092bc5f094 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:39.191435 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d285cd1-9ae8-453a-b2dc-bb092bc5f094 admin admin] PATH_INFO: `/v3/users/bda0864849c045199ee90cba6553d5ea` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:39.193586 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d285cd1-9ae8-453a-b2dc-bb092bc5f094 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bda0864849c045199ee90cba6553d5ea)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:39.196101 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d285cd1-9ae8-453a-b2dc-bb092bc5f094 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:39.253729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 872/1729] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:03:39 2022] POST /identity/v3/users => generated 295 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:39.257845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0f921-55f1-4c09-92ea-4a1a9555dbaf admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:39.259846 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-48c0f921-55f1-4c09-92ea-4a1a9555dbaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:39.273989 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0f921-55f1-4c09-92ea-4a1a9555dbaf admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:39.279161 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d6223-b1ee-4b60-8669-5796e22cd276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:39.280160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d6223-b1ee-4b60-8669-5796e22cd276 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:39.280160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d6223-b1ee-4b60-8669-5796e22cd276 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:39.280547 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d6223-b1ee-4b60-8669-5796e22cd276 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:39.280954 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d6223-b1ee-4b60-8669-5796e22cd276 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:39.283056 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d6223-b1ee-4b60-8669-5796e22cd276 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:39.453489 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 873/1730] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:03:39 2022] POST /identity/v3/projects => generated 373 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:39.456677 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d6223-b1ee-4b60-8669-5796e22cd276 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:39.643178 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d6223-b1ee-4b60-8669-5796e22cd276 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:39.644875 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d8e32c-ec0f-4a58-81da-fa68c5a12316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:39.645733 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8e32c-ec0f-4a58-81da-fa68c5a12316 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:39.646081 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8e32c-ec0f-4a58-81da-fa68c5a12316 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:39.646481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8e32c-ec0f-4a58-81da-fa68c5a12316 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:39.647173 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d8e32c-ec0f-4a58-81da-fa68c5a12316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:39.648954 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d8e32c-ec0f-4a58-81da-fa68c5a12316 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:39.661159 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 874/1731] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:39.662531 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d8e32c-ec0f-4a58-81da-fa68c5a12316 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:39.691441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-3d285cd1-9ae8-453a-b2dc-bb092bc5f094 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bda0864849c045199ee90cba6553d5ea'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:39.709594 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d8e32c-ec0f-4a58-81da-fa68c5a12316 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:39.724988 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df97428-a6c6-4c70-b958-b3db3b422bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:39.724988 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df97428-a6c6-4c70-b958-b3db3b422bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:39.724988 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df97428-a6c6-4c70-b958-b3db3b422bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:39.724988 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df97428-a6c6-4c70-b958-b3db3b422bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:39.725410 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df97428-a6c6-4c70-b958-b3db3b422bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:39.725410 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df97428-a6c6-4c70-b958-b3db3b422bd8 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:39.739469 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 875/1732] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:39.740724 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df97428-a6c6-4c70-b958-b3db3b422bd8 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:39.771379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df97428-a6c6-4c70-b958-b3db3b422bd8 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:39.776462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce0ea6-196e-43c5-a149-b3668a58bf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:39.777078 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce0ea6-196e-43c5-a149-b3668a58bf74 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:39.777358 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce0ea6-196e-43c5-a149-b3668a58bf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:39.777630 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce0ea6-196e-43c5-a149-b3668a58bf74 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:39.778271 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce0ea6-196e-43c5-a149-b3668a58bf74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:39.780376 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce0ea6-196e-43c5-a149-b3668a58bf74 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:39.909601 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-3d285cd1-9ae8-453a-b2dc-bb092bc5f094 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bda0864849c045199ee90cba6553d5ea'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:39.910769 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-3d285cd1-9ae8-453a-b2dc-bb092bc5f094 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bda0864849c045199ee90cba6553d5ea'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:39.911426 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-3d285cd1-9ae8-453a-b2dc-bb092bc5f094 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bda0864849c045199ee90cba6553d5ea'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:39.924789 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-3d285cd1-9ae8-453a-b2dc-bb092bc5f094 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bda0864849c045199ee90cba6553d5ea'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:39.937516 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 858/1733] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:03:39 2022] DELETE /identity/v3/users/bda0864849c045199ee90cba6553d5ea => generated 0 bytes in 788 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:39.938445 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d285cd1-9ae8-453a-b2dc-bb092bc5f094 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:39.942255 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-3d285cd1-9ae8-453a-b2dc-bb092bc5f094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:39.969161 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 876/1734] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:03:39 2022] POST /identity/v3/users => generated 287 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:39.974342 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62453e15-b26c-491f-a4e6-a46b9c3fb667 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:39.975121 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62453e15-b26c-491f-a4e6-a46b9c3fb667 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:39.975405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62453e15-b26c-491f-a4e6-a46b9c3fb667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:40.028524 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-62453e15-b26c-491f-a4e6-a46b9c3fb667 None None] MFA Rules not processed for user `5d1ce9150ba543b1b06f8edb5a545af0`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:40.035593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-62453e15-b26c-491f-a4e6-a46b9c3fb667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:40.071531 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:40.139679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d285cd1-9ae8-453a-b2dc-bb092bc5f094 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:40.150115 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c593e-4c30-42e2-9b61-b73fe11c5d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:40.151207 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c593e-4c30-42e2-9b61-b73fe11c5d3e admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:40.151682 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c593e-4c30-42e2-9b61-b73fe11c5d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:40.152267 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c593e-4c30-42e2-9b61-b73fe11c5d3e admin admin] PATH_INFO: `/v3/projects/04df4a9a868a4a66984212ee0bbd0a66/users/6c7b814a997d4809b0aa9080596e6e83/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:40.180452 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c593e-4c30-42e2-9b61-b73fe11c5d3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=04df4a9a868a4a66984212ee0bbd0a66, user_id=6c7b814a997d4809b0aa9080596e6e83, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:40.187363 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c593e-4c30-42e2-9b61-b73fe11c5d3e admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:40.223746 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 877/1735] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:40.223746 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62453e15-b26c-491f-a4e6-a46b9c3fb667 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:40.243643 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-62453e15-b26c-491f-a4e6-a46b9c3fb667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:40.284236 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62453e15-b26c-491f-a4e6-a46b9c3fb667 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:40.302216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aecb0d5-5043-4c34-89cb-9e241a0025ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:40.302968 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecb0d5-5043-4c34-89cb-9e241a0025ba admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:40.303245 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecb0d5-5043-4c34-89cb-9e241a0025ba admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:40.303516 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecb0d5-5043-4c34-89cb-9e241a0025ba admin admin] PATH_INFO: `/v3/projects/e02d08c7af744469a1d8a389b18896a3/users/49221778883942e0907f265bdf146495/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:40.336524 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 859/1736] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:39 2022] PUT /identity/v3/projects/04df4a9a868a4a66984212ee0bbd0a66/users/6c7b814a997d4809b0aa9080596e6e83/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:40.336988 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c593e-4c30-42e2-9b61-b73fe11c5d3e admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:40.348464 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aecb0d5-5043-4c34-89cb-9e241a0025ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=e02d08c7af744469a1d8a389b18896a3, user_id=49221778883942e0907f265bdf146495, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:40.349188 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-123c593e-4c30-42e2-9b61-b73fe11c5d3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:40.351636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aecb0d5-5043-4c34-89cb-9e241a0025ba admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:40.525492 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 878/1737] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:40 2022] PUT /identity/v3/projects/e02d08c7af744469a1d8a389b18896a3/users/49221778883942e0907f265bdf146495/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:40.527347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117af24-349c-4e96-a492-f7c4b7cd2a32 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:40.528743 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117af24-349c-4e96-a492-f7c4b7cd2a32 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:40.530653 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117af24-349c-4e96-a492-f7c4b7cd2a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:40.555600 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c593e-4c30-42e2-9b61-b73fe11c5d3e admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:40.623290 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-4117af24-349c-4e96-a492-f7c4b7cd2a32 None None] MFA Rules not processed for user `6c7b814a997d4809b0aa9080596e6e83`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:40.624642 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-4117af24-349c-4e96-a492-f7c4b7cd2a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:40.797906 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b1d6e-3bc8-4db1-82d2-73516f04314c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:40.799331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b1d6e-3bc8-4db1-82d2-73516f04314c admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:40.799766 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b1d6e-3bc8-4db1-82d2-73516f04314c admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:40.800046 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b1d6e-3bc8-4db1-82d2-73516f04314c admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:40.800712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b1d6e-3bc8-4db1-82d2-73516f04314c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:40.805342 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b1d6e-3bc8-4db1-82d2-73516f04314c admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:40.828998 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 860/1738] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 492 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:40.830925 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b1d6e-3bc8-4db1-82d2-73516f04314c admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:40.849546 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b1d6e-3bc8-4db1-82d2-73516f04314c admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:40.874716 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 879/1739] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:40.878275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4117af24-349c-4e96-a492-f7c4b7cd2a32 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:40.889541 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e284f-3315-4114-9ab2-85da187ede38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:40.889541 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e284f-3315-4114-9ab2-85da187ede38 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:40.889541 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e284f-3315-4114-9ab2-85da187ede38 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:40.889541 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e284f-3315-4114-9ab2-85da187ede38 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:40.900479 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e284f-3315-4114-9ab2-85da187ede38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:40.903178 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4117af24-349c-4e96-a492-f7c4b7cd2a32 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:40.905128 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e284f-3315-4114-9ab2-85da187ede38 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:40.929974 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f7f53-a51c-4e00-a7e9-e42bd55fba57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:40.929974 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f7f53-a51c-4e00-a7e9-e42bd55fba57 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:40.930812 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f7f53-a51c-4e00-a7e9-e42bd55fba57 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:40.931945 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f7f53-a51c-4e00-a7e9-e42bd55fba57 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:40.933473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f7f53-a51c-4e00-a7e9-e42bd55fba57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:40.939677 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f7f53-a51c-4e00-a7e9-e42bd55fba57 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:40.942452 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 861/1740] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:40.944004 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e284f-3315-4114-9ab2-85da187ede38 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:40.958918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e284f-3315-4114-9ab2-85da187ede38 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:40.966420 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ab9c1-8390-431d-bf47-dcd9de870de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:40.971827 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ab9c1-8390-431d-bf47-dcd9de870de2 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:40.971827 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ab9c1-8390-431d-bf47-dcd9de870de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:40.971827 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ab9c1-8390-431d-bf47-dcd9de870de2 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:40.971827 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ab9c1-8390-431d-bf47-dcd9de870de2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:40.972595 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ab9c1-8390-431d-bf47-dcd9de870de2 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:40.998876 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 880/1741] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:41.000536 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f7f53-a51c-4e00-a7e9-e42bd55fba57 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:41.040979 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f7f53-a51c-4e00-a7e9-e42bd55fba57 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:41.062349 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1deba-7475-47c4-8712-5ebd37f23da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:41.063104 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1deba-7475-47c4-8712-5ebd37f23da5 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:41.063392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1deba-7475-47c4-8712-5ebd37f23da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:41.063700 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1deba-7475-47c4-8712-5ebd37f23da5 admin admin] PATH_INFO: `/v3/projects/e02d08c7af744469a1d8a389b18896a3/users/49221778883942e0907f265bdf146495/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:41.073925 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea1deba-7475-47c4-8712-5ebd37f23da5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e02d08c7af744469a1d8a389b18896a3, user_id=49221778883942e0907f265bdf146495, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:41.076862 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea1deba-7475-47c4-8712-5ebd37f23da5 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:41.113138 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 881/1742] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:40 2022] PUT /identity/v3/projects/e02d08c7af744469a1d8a389b18896a3/users/49221778883942e0907f265bdf146495/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:41.114541 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1deba-7475-47c4-8712-5ebd37f23da5 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:41.210878 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 862/1743] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:03:40 2022] POST /identity/v3/projects => generated 363 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:41.212271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab73ab4-4950-4751-b11b-8e3e5276d293 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:41.212822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab73ab4-4950-4751-b11b-8e3e5276d293 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:41.215181 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab73ab4-4950-4751-b11b-8e3e5276d293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:41.273303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-eab73ab4-4950-4751-b11b-8e3e5276d293 None None] MFA Rules not processed for user `49221778883942e0907f265bdf146495`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:41.275370 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-eab73ab4-4950-4751-b11b-8e3e5276d293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:41.282156 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1deba-7475-47c4-8712-5ebd37f23da5 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:41.410285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 863/1744] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:41.411577 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab73ab4-4950-4751-b11b-8e3e5276d293 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:41.457060 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa1dee-daef-4c43-9c9e-3a313328d89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:41.465472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa1dee-daef-4c43-9c9e-3a313328d89c admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:41.465472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa1dee-daef-4c43-9c9e-3a313328d89c admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:41.465472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa1dee-daef-4c43-9c9e-3a313328d89c admin admin] PATH_INFO: `/v3/projects/d87c66953a6e4022b47a0fc0383cf768/users/50502c41682b4bd9a931cdb7d7f4f12f/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:41.578791 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa1dee-daef-4c43-9c9e-3a313328d89c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d87c66953a6e4022b47a0fc0383cf768, user_id=50502c41682b4bd9a931cdb7d7f4f12f, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:41.582817 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa1dee-daef-4c43-9c9e-3a313328d89c admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:41.606370 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab73ab4-4950-4751-b11b-8e3e5276d293 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:41.620866 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c88b2-9614-4e21-8cc5-ba761a8c96ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:41.621531 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c88b2-9614-4e21-8cc5-ba761a8c96ed admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:41.622337 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c88b2-9614-4e21-8cc5-ba761a8c96ed admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:41.622337 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c88b2-9614-4e21-8cc5-ba761a8c96ed admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:41.628174 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c88b2-9614-4e21-8cc5-ba761a8c96ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:41.629527 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c88b2-9614-4e21-8cc5-ba761a8c96ed admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:41.694876 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 882/1745] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:41 2022] PUT /identity/v3/projects/d87c66953a6e4022b47a0fc0383cf768/users/50502c41682b4bd9a931cdb7d7f4f12f/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:41.700683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa1dee-daef-4c43-9c9e-3a313328d89c admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:41.704193 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-dcfa1dee-daef-4c43-9c9e-3a313328d89c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:41.757156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 864/1746] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:03:41 2022] POST /identity/v3/users => generated 288 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:41.765704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c88b2-9614-4e21-8cc5-ba761a8c96ed admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:41.765704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-148c88b2-9614-4e21-8cc5-ba761a8c96ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:41.815360 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa1dee-daef-4c43-9c9e-3a313328d89c admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:41.823965 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905d04d1-1efd-4f7c-a33a-34f0bbbc6219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:41.824713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905d04d1-1efd-4f7c-a33a-34f0bbbc6219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:41.825003 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905d04d1-1efd-4f7c-a33a-34f0bbbc6219 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:41.825282 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905d04d1-1efd-4f7c-a33a-34f0bbbc6219 admin admin] PATH_INFO: `/v3/projects/ad6bd7f10d714fd9a6697ec06fbc72a4` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:41.826537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905d04d1-1efd-4f7c-a33a-34f0bbbc6219 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad6bd7f10d714fd9a6697ec06fbc72a4)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:41.828757 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905d04d1-1efd-4f7c-a33a-34f0bbbc6219 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:41.862742 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c88b2-9614-4e21-8cc5-ba761a8c96ed admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:41.933357 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476454b-073c-4e5c-89b2-1c1a7792d3e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:41.934004 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476454b-073c-4e5c-89b2-1c1a7792d3e6 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:41.934286 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476454b-073c-4e5c-89b2-1c1a7792d3e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:41.934637 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476454b-073c-4e5c-89b2-1c1a7792d3e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:41.936485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-f476454b-073c-4e5c-89b2-1c1a7792d3e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:42.058801 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f476454b-073c-4e5c-89b2-1c1a7792d3e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:42.060941 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f476454b-073c-4e5c-89b2-1c1a7792d3e6 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:42.086889 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-905d04d1-1efd-4f7c-a33a-34f0bbbc6219 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad6bd7f10d714fd9a6697ec06fbc72a4', 'request_id': 'req-905d04d1-1efd-4f7c-a33a-34f0bbbc6219'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:42.091856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 865/1747] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:42.096679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514b5bd-f2be-4666-aab9-6070a1eb820a None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:42.097244 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514b5bd-f2be-4666-aab9-6070a1eb820a None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:42.098164 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514b5bd-f2be-4666-aab9-6070a1eb820a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:42.139625 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-8514b5bd-f2be-4666-aab9-6070a1eb820a None None] MFA Rules not processed for user `50502c41682b4bd9a931cdb7d7f4f12f`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:42.141554 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-8514b5bd-f2be-4666-aab9-6070a1eb820a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:42.196473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-905d04d1-1efd-4f7c-a33a-34f0bbbc6219 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad6bd7f10d714fd9a6697ec06fbc72a4', 'request_id': 'req-905d04d1-1efd-4f7c-a33a-34f0bbbc6219'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:42.206509 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-905d04d1-1efd-4f7c-a33a-34f0bbbc6219 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad6bd7f10d714fd9a6697ec06fbc72a4', 'request_id': 'req-905d04d1-1efd-4f7c-a33a-34f0bbbc6219'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:42.231383 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-905d04d1-1efd-4f7c-a33a-34f0bbbc6219 admin admin] The token cache is being invalidate because project ad6bd7f10d714fd9a6697ec06fbc72a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:03:42.231966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-905d04d1-1efd-4f7c-a33a-34f0bbbc6219 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:42.233245 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-905d04d1-1efd-4f7c-a33a-34f0bbbc6219 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:42.235628 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 883/1748] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:03:41 2022] DELETE /identity/v3/projects/ad6bd7f10d714fd9a6697ec06fbc72a4 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:42.245851 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905d04d1-1efd-4f7c-a33a-34f0bbbc6219 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:42.262788 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-905d04d1-1efd-4f7c-a33a-34f0bbbc6219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:42.322019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 866/1749] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:42.323534 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1388f73-884a-4db4-a788-dac470e08ab8 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:42.323820 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1388f73-884a-4db4-a788-dac470e08ab8 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:42.324096 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1388f73-884a-4db4-a788-dac470e08ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:42.351841 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-a1388f73-884a-4db4-a788-dac470e08ab8 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:42.353553 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a1388f73-884a-4db4-a788-dac470e08ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:42.457936 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 867/1750] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:42.458933 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1388f73-884a-4db4-a788-dac470e08ab8 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:42.461588 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a1388f73-884a-4db4-a788-dac470e08ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:42.471086 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905d04d1-1efd-4f7c-a33a-34f0bbbc6219 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:42.476314 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89788308-7940-48da-9dd7-ecf6e2fb5bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:42.476932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89788308-7940-48da-9dd7-ecf6e2fb5bfc admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:42.477211 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89788308-7940-48da-9dd7-ecf6e2fb5bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:42.477484 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89788308-7940-48da-9dd7-ecf6e2fb5bfc admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:42.478125 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89788308-7940-48da-9dd7-ecf6e2fb5bfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:42.480718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89788308-7940-48da-9dd7-ecf6e2fb5bfc admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:42.493226 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 884/1751] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:42.496862 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6435a-4147-46e0-90d4-7813c94961eb None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:42.497149 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6435a-4147-46e0-90d4-7813c94961eb None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:42.497627 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6435a-4147-46e0-90d4-7813c94961eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:42.518244 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-e0e6435a-4147-46e0-90d4-7813c94961eb None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:42.519441 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-e0e6435a-4147-46e0-90d4-7813c94961eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:42.531827 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 885/1752] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:42.532818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6435a-4147-46e0-90d4-7813c94961eb None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:42.545146 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6435a-4147-46e0-90d4-7813c94961eb None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:42.550175 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ce6c2-e188-4c25-841b-deb446311260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:42.550879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ce6c2-e188-4c25-841b-deb446311260 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:42.551172 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ce6c2-e188-4c25-841b-deb446311260 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:42.551474 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ce6c2-e188-4c25-841b-deb446311260 admin admin] PATH_INFO: `/v3/projects/65bf846fe87e4165ad3f6d46641cfcb9/users/5066d7f1d7044ee6a71b63a526845c08/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:42.568141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ce6c2-e188-4c25-841b-deb446311260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65bf846fe87e4165ad3f6d46641cfcb9, user_id=5066d7f1d7044ee6a71b63a526845c08, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:42.570817 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ce6c2-e188-4c25-841b-deb446311260 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:42.579130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1388f73-884a-4db4-a788-dac470e08ab8 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:42.584796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97d7e9-c711-4b93-a7d1-5cb11f1bb570 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:42.585436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97d7e9-c711-4b93-a7d1-5cb11f1bb570 service nova] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:42.585715 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97d7e9-c711-4b93-a7d1-5cb11f1bb570 service nova] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:42.586102 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97d7e9-c711-4b93-a7d1-5cb11f1bb570 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:42.594682 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d97d7e9-c711-4b93-a7d1-5cb11f1bb570 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:42.596491 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d97d7e9-c711-4b93-a7d1-5cb11f1bb570 service nova] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:42.608235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 868/1753] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:42.611089 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97d7e9-c711-4b93-a7d1-5cb11f1bb570 service nova] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:42.631722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97d7e9-c711-4b93-a7d1-5cb11f1bb570 service nova] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:42.655702 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ddfc0-0ee5-4f1a-802b-f645e350c406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:42.656335 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ddfc0-0ee5-4f1a-802b-f645e350c406 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:42.656613 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ddfc0-0ee5-4f1a-802b-f645e350c406 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:42.656888 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ddfc0-0ee5-4f1a-802b-f645e350c406 admin admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:42.657543 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268ddfc0-0ee5-4f1a-802b-f645e350c406 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:03:42.657842 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268ddfc0-0ee5-4f1a-802b-f645e350c406 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:42.662758 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268ddfc0-0ee5-4f1a-802b-f645e350c406 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:42.686347 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 869/1754] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:03:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:03:42.690882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ddfc0-0ee5-4f1a-802b-f645e350c406 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:42.711872 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ddfc0-0ee5-4f1a-802b-f645e350c406 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:42.721110 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebbafc6-5458-4537-8b2e-2f51fd3e6a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:42.721848 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbafc6-5458-4537-8b2e-2f51fd3e6a31 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:42.722126 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbafc6-5458-4537-8b2e-2f51fd3e6a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:42.722482 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbafc6-5458-4537-8b2e-2f51fd3e6a31 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:42.723662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebbafc6-5458-4537-8b2e-2f51fd3e6a31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:42.729688 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebbafc6-5458-4537-8b2e-2f51fd3e6a31 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:42.730016 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 886/1755] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:42 2022] PUT /identity/v3/projects/65bf846fe87e4165ad3f6d46641cfcb9/users/5066d7f1d7044ee6a71b63a526845c08/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:42.731221 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ce6c2-e188-4c25-841b-deb446311260 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:42.834674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ce6c2-e188-4c25-841b-deb446311260 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:42.842471 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf6863-bf6c-4621-85fb-4f0afafb38aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:42.843226 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf6863-bf6c-4621-85fb-4f0afafb38aa admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:42.843518 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf6863-bf6c-4621-85fb-4f0afafb38aa admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:42.843929 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf6863-bf6c-4621-85fb-4f0afafb38aa admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:42.844462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf6863-bf6c-4621-85fb-4f0afafb38aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:42.847494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf6863-bf6c-4621-85fb-4f0afafb38aa admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:42.880669 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 887/1756] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:42.882536 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf6863-bf6c-4621-85fb-4f0afafb38aa admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:42.899124 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf6863-bf6c-4621-85fb-4f0afafb38aa admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:42.908463 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf0194-9333-465d-a59a-7956230bf9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:42.908463 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf0194-9333-465d-a59a-7956230bf9b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:42.908819 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf0194-9333-465d-a59a-7956230bf9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:42.910099 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 870/1757] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:03:42 2022] POST /identity/v3/projects => generated 369 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:42.910312 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf0194-9333-465d-a59a-7956230bf9b9 admin admin] PATH_INFO: `/v3/projects/65bf846fe87e4165ad3f6d46641cfcb9/users/5066d7f1d7044ee6a71b63a526845c08/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:42.911305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebbafc6-5458-4537-8b2e-2f51fd3e6a31 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:42.912524 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf0194-9333-465d-a59a-7956230bf9b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65bf846fe87e4165ad3f6d46641cfcb9, user_id=5066d7f1d7044ee6a71b63a526845c08, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:42.915659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf0194-9333-465d-a59a-7956230bf9b9 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:42.934482 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 888/1758] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:42 2022] PUT /identity/v3/projects/65bf846fe87e4165ad3f6d46641cfcb9/users/5066d7f1d7044ee6a71b63a526845c08/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:42.936398 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf0194-9333-465d-a59a-7956230bf9b9 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:42.939016 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2faf0194-9333-465d-a59a-7956230bf9b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:42.980408 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:DELETE=8,keystone:SELECT=92 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:43.013493 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebbafc6-5458-4537-8b2e-2f51fd3e6a31 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:43.022174 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d14e3f2-b147-4da6-8364-21490a5e0443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:43.022904 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14e3f2-b147-4da6-8364-21490a5e0443 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:43.023319 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14e3f2-b147-4da6-8364-21490a5e0443 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:43.023634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14e3f2-b147-4da6-8364-21490a5e0443 admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:43.029377 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d14e3f2-b147-4da6-8364-21490a5e0443 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:43.033868 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d14e3f2-b147-4da6-8364-21490a5e0443 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:43.039892 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf0194-9333-465d-a59a-7956230bf9b9 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:43.052497 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4cc839-127d-4004-b9ac-8b2aef5c96f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:43.053147 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cc839-127d-4004-b9ac-8b2aef5c96f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:43.053423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cc839-127d-4004-b9ac-8b2aef5c96f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:43.053697 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cc839-127d-4004-b9ac-8b2aef5c96f5 admin admin] PATH_INFO: `/v3/users/50502c41682b4bd9a931cdb7d7f4f12f` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:43.057564 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4cc839-127d-4004-b9ac-8b2aef5c96f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50502c41682b4bd9a931cdb7d7f4f12f)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:43.059585 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4cc839-127d-4004-b9ac-8b2aef5c96f5 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:43.214492 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 871/1759] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:03:42 2022] POST /identity/v3/users => generated 285 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:43.220831 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c79510-18f4-4a75-9995-a48c3f4c7eb6 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:43.221150 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c79510-18f4-4a75-9995-a48c3f4c7eb6 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:43.221437 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c79510-18f4-4a75-9995-a48c3f4c7eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:43.267703 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-78c79510-18f4-4a75-9995-a48c3f4c7eb6 None None] MFA Rules not processed for user `5066d7f1d7044ee6a71b63a526845c08`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:43.269434 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-78c79510-18f4-4a75-9995-a48c3f4c7eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:43.363087 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 872/1760] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:43 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:43.364220 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c79510-18f4-4a75-9995-a48c3f4c7eb6 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:43.367339 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-7b4cc839-127d-4004-b9ac-8b2aef5c96f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50502c41682b4bd9a931cdb7d7f4f12f'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:43.383961 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-7b4cc839-127d-4004-b9ac-8b2aef5c96f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50502c41682b4bd9a931cdb7d7f4f12f'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:43.392524 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c79510-18f4-4a75-9995-a48c3f4c7eb6 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:43.396610 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-7b4cc839-127d-4004-b9ac-8b2aef5c96f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50502c41682b4bd9a931cdb7d7f4f12f'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:43.402058 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5e4ff9-7cb8-43ee-be9a-c68d83fc986a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:43.415496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e4ff9-7cb8-43ee-be9a-c68d83fc986a admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:43.415785 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e4ff9-7cb8-43ee-be9a-c68d83fc986a admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:43.416063 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e4ff9-7cb8-43ee-be9a-c68d83fc986a admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:43.416851 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5e4ff9-7cb8-43ee-be9a-c68d83fc986a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:43.419710 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5e4ff9-7cb8-43ee-be9a-c68d83fc986a admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:43.434072 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 873/1761] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:43.437965 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5e4ff9-7cb8-43ee-be9a-c68d83fc986a admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:43.439820 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-8f5e4ff9-7cb8-43ee-be9a-c68d83fc986a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:43.539279 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5e4ff9-7cb8-43ee-be9a-c68d83fc986a admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:43.539760 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-7b4cc839-127d-4004-b9ac-8b2aef5c96f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50502c41682b4bd9a931cdb7d7f4f12f'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:43.540678 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-7b4cc839-127d-4004-b9ac-8b2aef5c96f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50502c41682b4bd9a931cdb7d7f4f12f'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:43.544267 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab88cfc-e075-4427-be81-555582dc34dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:43.548850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab88cfc-e075-4427-be81-555582dc34dd service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:43.548850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab88cfc-e075-4427-be81-555582dc34dd service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:43.548850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab88cfc-e075-4427-be81-555582dc34dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:43.548850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-6ab88cfc-e075-4427-be81-555582dc34dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:43.552031 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 889/1762] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:03:42 2022] DELETE /identity/v3/users/50502c41682b4bd9a931cdb7d7f4f12f => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:43.553094 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4cc839-127d-4004-b9ac-8b2aef5c96f5 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:43.555738 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-7b4cc839-127d-4004-b9ac-8b2aef5c96f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:43.572035 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab88cfc-e075-4427-be81-555582dc34dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:43.574674 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab88cfc-e075-4427-be81-555582dc34dd service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:43.623184 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 874/1763] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:43 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:43.637956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9808a-9622-4076-83b0-38fdef38434d None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:43.638223 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9808a-9622-4076-83b0-38fdef38434d None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:43.641586 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9808a-9622-4076-83b0-38fdef38434d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:43.685976 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-cee9808a-9622-4076-83b0-38fdef38434d None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:43.687259 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-cee9808a-9622-4076-83b0-38fdef38434d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:43.738319 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4cc839-127d-4004-b9ac-8b2aef5c96f5 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:43.774851 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441eff4d-ef4c-4f9f-ad40-247bb597866f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:43.776816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 875/1764] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:43.778657 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441eff4d-ef4c-4f9f-ad40-247bb597866f admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:43.783791 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441eff4d-ef4c-4f9f-ad40-247bb597866f admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:43.783791 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441eff4d-ef4c-4f9f-ad40-247bb597866f admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:43.783791 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441eff4d-ef4c-4f9f-ad40-247bb597866f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:43.783791 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441eff4d-ef4c-4f9f-ad40-247bb597866f admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:43.792765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee9808a-9622-4076-83b0-38fdef38434d None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:43.794480 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-cee9808a-9622-4076-83b0-38fdef38434d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:43.796095 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 890/1765] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:43.800410 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441eff4d-ef4c-4f9f-ad40-247bb597866f admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:43.844430 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441eff4d-ef4c-4f9f-ad40-247bb597866f admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:43.861850 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773946eb-e856-4e63-ad55-399304c9996a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:43.863758 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773946eb-e856-4e63-ad55-399304c9996a admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:43.864172 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773946eb-e856-4e63-ad55-399304c9996a admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:43.864314 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773946eb-e856-4e63-ad55-399304c9996a admin admin] PATH_INFO: `/v3/projects/e284aaf67cad42faa415a41b0097e82e/users/12c648102609454aa480eb68b15dcda1/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:43.919277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773946eb-e856-4e63-ad55-399304c9996a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e284aaf67cad42faa415a41b0097e82e, user_id=12c648102609454aa480eb68b15dcda1, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:43.922512 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773946eb-e856-4e63-ad55-399304c9996a admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:43.967649 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee9808a-9622-4076-83b0-38fdef38434d None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:43.973481 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06e048-322b-445c-94d6-aa9c0726c5c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:43.974115 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06e048-322b-445c-94d6-aa9c0726c5c1 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:43.974423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06e048-322b-445c-94d6-aa9c0726c5c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:43.974699 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06e048-322b-445c-94d6-aa9c0726c5c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:43.987231 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b06e048-322b-445c-94d6-aa9c0726c5c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:43.990020 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b06e048-322b-445c-94d6-aa9c0726c5c1 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:44.034612 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 891/1766] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:43 2022] PUT /identity/v3/projects/e284aaf67cad42faa415a41b0097e82e/users/12c648102609454aa480eb68b15dcda1/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:03:44.040031 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773946eb-e856-4e63-ad55-399304c9996a admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:44.044240 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-773946eb-e856-4e63-ad55-399304c9996a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:44.045857 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 876/1767] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:03:44.049123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9fb03-beaf-4cea-a9a7-5b0504504913 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:44.050269 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9fb03-beaf-4cea-a9a7-5b0504504913 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:44.050592 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9fb03-beaf-4cea-a9a7-5b0504504913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:44.085851 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-a5a9fb03-beaf-4cea-a9a7-5b0504504913 None None] MFA Rules not processed for user `12c648102609454aa480eb68b15dcda1`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:44.087113 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a5a9fb03-beaf-4cea-a9a7-5b0504504913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:44.113617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773946eb-e856-4e63-ad55-399304c9996a admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:44.117584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe17441-d98e-4690-8723-e9ff50abf797 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:44.118202 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe17441-d98e-4690-8723-e9ff50abf797 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:44.118512 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe17441-d98e-4690-8723-e9ff50abf797 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:44.118784 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe17441-d98e-4690-8723-e9ff50abf797 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:44.122196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-ffe17441-d98e-4690-8723-e9ff50abf797 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:44.176514 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 877/1768] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:44.181159 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9fb03-beaf-4cea-a9a7-5b0504504913 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:44.187584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe17441-d98e-4690-8723-e9ff50abf797 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:44.189025 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe17441-d98e-4690-8723-e9ff50abf797 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:44.201058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 892/1769] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:44.285277 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9fb03-beaf-4cea-a9a7-5b0504504913 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:44.290303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956529e-8e87-4430-b7ef-acc6d0cab8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:44.291002 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956529e-8e87-4430-b7ef-acc6d0cab8dd admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:44.291622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956529e-8e87-4430-b7ef-acc6d0cab8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:44.291901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956529e-8e87-4430-b7ef-acc6d0cab8dd admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:44.292550 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956529e-8e87-4430-b7ef-acc6d0cab8dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:44.294865 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956529e-8e87-4430-b7ef-acc6d0cab8dd admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:44.387207 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:44.437718 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 878/1770] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:03:44 2022] POST /identity/v3/projects => generated 365 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:03:44.443034 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe17441-d98e-4690-8723-e9ff50abf797 service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:44.460180 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe17441-d98e-4690-8723-e9ff50abf797 service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:44.557889 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303b0ef-0618-4948-aa38-c6d70edc80ce None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:44.557889 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303b0ef-0618-4948-aa38-c6d70edc80ce None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:44.557889 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303b0ef-0618-4948-aa38-c6d70edc80ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:44.608147 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d9832-1fb9-4625-a15b-be64ef2aadf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:44.608770 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d9832-1fb9-4625-a15b-be64ef2aadf0 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:44.609042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d9832-1fb9-4625-a15b-be64ef2aadf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:44.609312 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d9832-1fb9-4625-a15b-be64ef2aadf0 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:44.609955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43d9832-1fb9-4625-a15b-be64ef2aadf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:44.611937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43d9832-1fb9-4625-a15b-be64ef2aadf0 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:44.671585 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-8303b0ef-0618-4948-aa38-c6d70edc80ce None None] MFA Rules not processed for user `7796ebff70054de090e02c58bf7000f4`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:44.673937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-8303b0ef-0618-4948-aa38-c6d70edc80ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:44.784906 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 879/1771] 10.222.0.16 () {58 vars in 1009 bytes} [Wed Aug 31 11:03:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:44.812732 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 893/1772] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:03:44 2022] POST /identity/v3/users => generated 289 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:44.814554 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303b0ef-0618-4948-aa38-c6d70edc80ce None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:44.837188 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303b0ef-0618-4948-aa38-c6d70edc80ce None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:44.870071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee15de-81d4-4ac9-837b-21978305f3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:44.870999 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee15de-81d4-4ac9-837b-21978305f3f6 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:44.871405 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee15de-81d4-4ac9-837b-21978305f3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:44.871907 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee15de-81d4-4ac9-837b-21978305f3f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:44.872620 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ee15de-81d4-4ac9-837b-21978305f3f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:44.874431 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ee15de-81d4-4ac9-837b-21978305f3f6 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:44.886642 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 880/1773] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:44.890754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d9832-1fb9-4625-a15b-be64ef2aadf0 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:44.902657 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d9832-1fb9-4625-a15b-be64ef2aadf0 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:44.908396 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a207b2f5-78e4-4a92-8650-e827f2626f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:44.909986 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207b2f5-78e4-4a92-8650-e827f2626f4e admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:44.910556 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207b2f5-78e4-4a92-8650-e827f2626f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:44.911038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207b2f5-78e4-4a92-8650-e827f2626f4e admin admin] PATH_INFO: `/v3/projects/1ab100be58bf4896a0c8c04befc9a072/users/fd5f32a0749c47f49568d2b7d25217ad/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:44.933157 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207b2f5-78e4-4a92-8650-e827f2626f4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ab100be58bf4896a0c8c04befc9a072, user_id=fd5f32a0749c47f49568d2b7d25217ad, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:44.936257 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207b2f5-78e4-4a92-8650-e827f2626f4e admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:45.100190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 894/1774] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:44 2022] PUT /identity/v3/projects/1ab100be58bf4896a0c8c04befc9a072/users/fd5f32a0749c47f49568d2b7d25217ad/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:45.101021 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee15de-81d4-4ac9-837b-21978305f3f6 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:45.132324 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee15de-81d4-4ac9-837b-21978305f3f6 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:45.264587 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f17f622-ba99-4121-a839-da10460d342b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:45.265615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17f622-ba99-4121-a839-da10460d342b admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:45.265615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17f622-ba99-4121-a839-da10460d342b admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:45.266435 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17f622-ba99-4121-a839-da10460d342b admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:45.266781 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f17f622-ba99-4121-a839-da10460d342b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:45.268477 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f17f622-ba99-4121-a839-da10460d342b admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:45.281271 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a207b2f5-78e4-4a92-8650-e827f2626f4e admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:45.286462 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 881/1775] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:03:45.287817 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a207b2f5-78e4-4a92-8650-e827f2626f4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:45.288093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f17f622-ba99-4121-a839-da10460d342b admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:45.299971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f17f622-ba99-4121-a839-da10460d342b admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:45.306903 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418317a7-fd70-4a87-8e75-ede7051a5d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:45.308188 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418317a7-fd70-4a87-8e75-ede7051a5d3d admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:45.308564 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418317a7-fd70-4a87-8e75-ede7051a5d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:45.308846 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418317a7-fd70-4a87-8e75-ede7051a5d3d admin admin] PATH_INFO: `/v3/projects/1ab100be58bf4896a0c8c04befc9a072/users/fd5f32a0749c47f49568d2b7d25217ad/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:45.310657 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a207b2f5-78e4-4a92-8650-e827f2626f4e admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:45.313010 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418317a7-fd70-4a87-8e75-ede7051a5d3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ab100be58bf4896a0c8c04befc9a072, user_id=fd5f32a0749c47f49568d2b7d25217ad, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:45.315939 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418317a7-fd70-4a87-8e75-ede7051a5d3d admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:45.317031 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217645a8-7fc8-4f26-8c47-cdd29008f6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:45.317879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217645a8-7fc8-4f26-8c47-cdd29008f6b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:45.318332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217645a8-7fc8-4f26-8c47-cdd29008f6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:45.318803 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217645a8-7fc8-4f26-8c47-cdd29008f6b4 admin admin] PATH_INFO: `/v3/projects/d87c66953a6e4022b47a0fc0383cf768` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:45.320270 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217645a8-7fc8-4f26-8c47-cdd29008f6b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d87c66953a6e4022b47a0fc0383cf768)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:45.322467 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217645a8-7fc8-4f26-8c47-cdd29008f6b4 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:45.341311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 882/1776] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:45 2022] PUT /identity/v3/projects/1ab100be58bf4896a0c8c04befc9a072/users/fd5f32a0749c47f49568d2b7d25217ad/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:45.347535 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418317a7-fd70-4a87-8e75-ede7051a5d3d admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:45.356107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-418317a7-fd70-4a87-8e75-ede7051a5d3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:45.502987 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-217645a8-7fc8-4f26-8c47-cdd29008f6b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd87c66953a6e4022b47a0fc0383cf768', 'request_id': 'req-217645a8-7fc8-4f26-8c47-cdd29008f6b4'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:45.519624 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418317a7-fd70-4a87-8e75-ede7051a5d3d admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:45.606950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14b060-d5ab-4326-83ec-1adabfadf26d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8abc7c439a04622896b34ed0a7b8492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:45.609313 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14b060-d5ab-4326-83ec-1adabfadf26d service swift] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:45.609611 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14b060-d5ab-4326-83ec-1adabfadf26d service swift] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:45.610187 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14b060-d5ab-4326-83ec-1adabfadf26d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:45.680833 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-217645a8-7fc8-4f26-8c47-cdd29008f6b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd87c66953a6e4022b47a0fc0383cf768', 'request_id': 'req-217645a8-7fc8-4f26-8c47-cdd29008f6b4'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:45.693794 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14b060-d5ab-4326-83ec-1adabfadf26d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:45.696399 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14b060-d5ab-4326-83ec-1adabfadf26d service swift] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:45.710984 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 883/1777] 10.222.0.16 () {60 vars in 1384 bytes} [Wed Aug 31 11:03:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:03:45.716996 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcae3c-00e6-4666-b5de-46b202cfaaa9 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:45.717323 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcae3c-00e6-4666-b5de-46b202cfaaa9 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:45.717599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcae3c-00e6-4666-b5de-46b202cfaaa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:45.721976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-217645a8-7fc8-4f26-8c47-cdd29008f6b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd87c66953a6e4022b47a0fc0383cf768', 'request_id': 'req-217645a8-7fc8-4f26-8c47-cdd29008f6b4'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:45.771291 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-217645a8-7fc8-4f26-8c47-cdd29008f6b4 admin admin] The token cache is being invalidate because project d87c66953a6e4022b47a0fc0383cf768 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:03:45.771485 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-217645a8-7fc8-4f26-8c47-cdd29008f6b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:45.780328 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-217645a8-7fc8-4f26-8c47-cdd29008f6b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:45.784004 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 895/1778] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:03:45 2022] DELETE /identity/v3/projects/d87c66953a6e4022b47a0fc0383cf768 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:45.790335 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b3639-6286-45c7-95db-6c59065fdbb4 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:45.790918 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b3639-6286-45c7-95db-6c59065fdbb4 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:45.793142 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b3639-6286-45c7-95db-6c59065fdbb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:45.827806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-7cdcae3c-00e6-4666-b5de-46b202cfaaa9 None None] MFA Rules not processed for user `fd5f32a0749c47f49568d2b7d25217ad`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:45.848231 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-7cdcae3c-00e6-4666-b5de-46b202cfaaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:45.863435 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-ba0b3639-6286-45c7-95db-6c59065fdbb4 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:45.864716 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-ba0b3639-6286-45c7-95db-6c59065fdbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:46.080225 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 896/1779] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:46.087081 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71d059-cdc3-45fc-aa1c-1301dac3e1e0 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:46.087177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71d059-cdc3-45fc-aa1c-1301dac3e1e0 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:46.087384 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71d059-cdc3-45fc-aa1c-1301dac3e1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:46.102965 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 884/1780] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:45 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:46.118165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdcae3c-00e6-4666-b5de-46b202cfaaa9 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:46.122841 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-7cdcae3c-00e6-4666-b5de-46b202cfaaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:46.156672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-cc71d059-cdc3-45fc-aa1c-1301dac3e1e0 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:46.158787 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-cc71d059-cdc3-45fc-aa1c-1301dac3e1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:46.183082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 897/1781] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:46.185899 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71d059-cdc3-45fc-aa1c-1301dac3e1e0 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:46.193911 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:46.207575 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71d059-cdc3-45fc-aa1c-1301dac3e1e0 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:46.221052 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81f455-a292-47a0-a7c7-bb2a2e108cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:46.222494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81f455-a292-47a0-a7c7-bb2a2e108cc2 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:46.223148 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81f455-a292-47a0-a7c7-bb2a2e108cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:46.223497 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81f455-a292-47a0-a7c7-bb2a2e108cc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:46.224343 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e81f455-a292-47a0-a7c7-bb2a2e108cc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:03:46.224648 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e81f455-a292-47a0-a7c7-bb2a2e108cc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:46.230893 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e81f455-a292-47a0-a7c7-bb2a2e108cc2 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:46.249505 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 898/1782] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:03:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:03:46.255435 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81f455-a292-47a0-a7c7-bb2a2e108cc2 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:46.281498 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81f455-a292-47a0-a7c7-bb2a2e108cc2 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:46.306769 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdcae3c-00e6-4666-b5de-46b202cfaaa9 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:46.307599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b4d9e-ccd0-4a6f-b359-ca91fb684187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:46.308404 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b4d9e-ccd0-4a6f-b359-ca91fb684187 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:46.308847 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b4d9e-ccd0-4a6f-b359-ca91fb684187 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:46.309280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b4d9e-ccd0-4a6f-b359-ca91fb684187 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:46.310073 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b4d9e-ccd0-4a6f-b359-ca91fb684187 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:46.312497 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b4d9e-ccd0-4a6f-b359-ca91fb684187 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:46.320401 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd1e4f1-4ca0-42e5-9409-2c226fe9353c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:46.324330 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1e4f1-4ca0-42e5-9409-2c226fe9353c service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:46.336783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1e4f1-4ca0-42e5-9409-2c226fe9353c service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:46.336783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1e4f1-4ca0-42e5-9409-2c226fe9353c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:46.356451 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd1e4f1-4ca0-42e5-9409-2c226fe9353c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:46.361991 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd1e4f1-4ca0-42e5-9409-2c226fe9353c service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:46.408081 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 885/1783] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:46 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:46.658935 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 899/1784] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:03:46 2022] POST /identity/v3/projects => generated 405 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:46.664096 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd1e4f1-4ca0-42e5-9409-2c226fe9353c service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:46.707652 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd1e4f1-4ca0-42e5-9409-2c226fe9353c service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:46.764446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca72d8-2653-44d7-a331-09f96d29ac80 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:46.765071 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca72d8-2653-44d7-a331-09f96d29ac80 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:46.765224 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca72d8-2653-44d7-a331-09f96d29ac80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:46.803278 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-68ca72d8-2653-44d7-a331-09f96d29ac80 None None] MFA Rules not processed for user `1a00b7f45b2e47b8a01048c36d00f742`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:46.804621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-68ca72d8-2653-44d7-a331-09f96d29ac80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:46.874018 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8378a30f-c8d1-4cda-96e6-5d02d0f9240c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:46.875248 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8378a30f-c8d1-4cda-96e6-5d02d0f9240c admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:46.875770 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8378a30f-c8d1-4cda-96e6-5d02d0f9240c admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:46.876234 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8378a30f-c8d1-4cda-96e6-5d02d0f9240c admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:46.877115 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8378a30f-c8d1-4cda-96e6-5d02d0f9240c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:46.879161 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8378a30f-c8d1-4cda-96e6-5d02d0f9240c admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:46.979320 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 900/1785] 10.222.0.69 () {58 vars in 1015 bytes} [Wed Aug 31 11:03:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:47.010956 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca72d8-2653-44d7-a331-09f96d29ac80 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:47.013607 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-68ca72d8-2653-44d7-a331-09f96d29ac80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:47.032405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca72d8-2653-44d7-a331-09f96d29ac80 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:47.043769 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f5f13-f499-406b-8a9e-b11c010e5cb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:47.046522 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f5f13-f499-406b-8a9e-b11c010e5cb2 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:47.047055 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f5f13-f499-406b-8a9e-b11c010e5cb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:47.047565 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f5f13-f499-406b-8a9e-b11c010e5cb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:47.066520 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f5f13-f499-406b-8a9e-b11c010e5cb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:47.071400 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f5f13-f499-406b-8a9e-b11c010e5cb2 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:47.102206 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 886/1786] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:03:46 2022] POST /identity/v3/users => generated 303 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:47.102206 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8378a30f-c8d1-4cda-96e6-5d02d0f9240c admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:47.114599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 901/1787] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:47.121064 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8378a30f-c8d1-4cda-96e6-5d02d0f9240c admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:47.137106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b96bd-1034-4f68-90ea-5ff47b5bcc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:47.140130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b96bd-1034-4f68-90ea-5ff47b5bcc2c admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:47.140416 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b96bd-1034-4f68-90ea-5ff47b5bcc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:47.140698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b96bd-1034-4f68-90ea-5ff47b5bcc2c admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:47.141362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6b96bd-1034-4f68-90ea-5ff47b5bcc2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:47.152157 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6b96bd-1034-4f68-90ea-5ff47b5bcc2c admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:47.189631 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f5f13-f499-406b-8a9e-b11c010e5cb2 service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:47.193725 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 887/1788] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:47.214045 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f5f13-f499-406b-8a9e-b11c010e5cb2 service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:47.243981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82393eb7-587f-4bf9-afff-ba3e38c69730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:47.245469 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82393eb7-587f-4bf9-afff-ba3e38c69730 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:47.245847 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82393eb7-587f-4bf9-afff-ba3e38c69730 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:47.246211 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82393eb7-587f-4bf9-afff-ba3e38c69730 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:47.247469 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82393eb7-587f-4bf9-afff-ba3e38c69730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:47.249295 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82393eb7-587f-4bf9-afff-ba3e38c69730 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:47.292764 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 902/1789] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:47.315294 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b96bd-1034-4f68-90ea-5ff47b5bcc2c admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:47.334139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b96bd-1034-4f68-90ea-5ff47b5bcc2c admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:47.351049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe9fb7-93db-460e-b5f3-7f0ab6da454f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:47.351715 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe9fb7-93db-460e-b5f3-7f0ab6da454f admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:47.352001 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe9fb7-93db-460e-b5f3-7f0ab6da454f admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:47.352440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe9fb7-93db-460e-b5f3-7f0ab6da454f admin admin] PATH_INFO: `/v3/projects/d48032da9f7f43a1954f462c2da1d408/users/542aa8bea90541f1b8e54bbfe423a392/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:47.396245 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fe9fb7-93db-460e-b5f3-7f0ab6da454f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d48032da9f7f43a1954f462c2da1d408, user_id=542aa8bea90541f1b8e54bbfe423a392, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:47.399293 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fe9fb7-93db-460e-b5f3-7f0ab6da454f admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:47.549268 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 888/1790] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:47 2022] PUT /identity/v3/projects/d48032da9f7f43a1954f462c2da1d408/users/542aa8bea90541f1b8e54bbfe423a392/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:47.561112 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a0f47-548a-4902-80af-f4909f290a75 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:47.561727 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a0f47-548a-4902-80af-f4909f290a75 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:47.562302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a0f47-548a-4902-80af-f4909f290a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:47.644425 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-fb9a0f47-548a-4902-80af-f4909f290a75 None None] MFA Rules not processed for user `542aa8bea90541f1b8e54bbfe423a392`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:47.646171 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-fb9a0f47-548a-4902-80af-f4909f290a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:47.803248 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 903/1791] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:47 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:47.811343 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe9fb7-93db-460e-b5f3-7f0ab6da454f admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:47.977114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe9fb7-93db-460e-b5f3-7f0ab6da454f admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:48.003103 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837bddc7-ae1d-4fdb-b819-95f8f6c4fea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:48.003732 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837bddc7-ae1d-4fdb-b819-95f8f6c4fea9 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:48.003998 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837bddc7-ae1d-4fdb-b819-95f8f6c4fea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:48.004269 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837bddc7-ae1d-4fdb-b819-95f8f6c4fea9 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:48.004917 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837bddc7-ae1d-4fdb-b819-95f8f6c4fea9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:48.007146 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837bddc7-ae1d-4fdb-b819-95f8f6c4fea9 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:48.267420 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a0f47-548a-4902-80af-f4909f290a75 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:48.273795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 889/1792] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:03:47 2022] POST /identity/v3/projects => generated 405 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:48.282614 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a0f47-548a-4902-80af-f4909f290a75 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:48.363745 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b9a07-6bea-4ac6-bebf-d04ed1086aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:48.363936 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b9a07-6bea-4ac6-bebf-d04ed1086aea admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:48.364452 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b9a07-6bea-4ac6-bebf-d04ed1086aea admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:48.364612 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b9a07-6bea-4ac6-bebf-d04ed1086aea admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:48.365324 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b9a07-6bea-4ac6-bebf-d04ed1086aea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:48.367423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b9a07-6bea-4ac6-bebf-d04ed1086aea admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:48.553696 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 904/1793] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:03:48 2022] POST /identity/v3/users => generated 309 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:48.570674 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837bddc7-ae1d-4fdb-b819-95f8f6c4fea9 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:48.584873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837bddc7-ae1d-4fdb-b819-95f8f6c4fea9 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:48.615943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd7b1a-ca68-4991-a214-414b012029f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:48.627081 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd7b1a-ca68-4991-a214-414b012029f5 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:48.627081 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd7b1a-ca68-4991-a214-414b012029f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:48.627081 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd7b1a-ca68-4991-a214-414b012029f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:48.627081 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd7b1a-ca68-4991-a214-414b012029f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:48.627081 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd7b1a-ca68-4991-a214-414b012029f5 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:48.633909 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 890/1794] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:48.639002 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b9a07-6bea-4ac6-bebf-d04ed1086aea admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:48.650969 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b9a07-6bea-4ac6-bebf-d04ed1086aea admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:48.656520 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc431644-4e43-458d-bce0-7580ae2fea93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:48.657177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc431644-4e43-458d-bce0-7580ae2fea93 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:48.657451 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc431644-4e43-458d-bce0-7580ae2fea93 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:48.657718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc431644-4e43-458d-bce0-7580ae2fea93 admin admin] PATH_INFO: `/v3/projects/7d928297537a4588ab57773cc8d43ff4/users/66d9603539dc496491fd108f4b73adfc/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:48.673407 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc431644-4e43-458d-bce0-7580ae2fea93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d928297537a4588ab57773cc8d43ff4, user_id=66d9603539dc496491fd108f4b73adfc, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:48.678645 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc431644-4e43-458d-bce0-7580ae2fea93 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:48.911791 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6056828-09d2-450c-91d2-1acc3893e006 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:48.912087 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6056828-09d2-450c-91d2-1acc3893e006 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:48.912613 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6056828-09d2-450c-91d2-1acc3893e006 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:48.914275 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 891/1795] 10.222.0.16 () {54 vars in 921 bytes} [Wed Aug 31 11:03:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:03:48.938029 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbf502-ab3e-42b0-9499-210a104a6cc3 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:48.938823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbf502-ab3e-42b0-9499-210a104a6cc3 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:48.939305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbf502-ab3e-42b0-9499-210a104a6cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:48.941220 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 905/1796] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:48 2022] PUT /identity/v3/projects/7d928297537a4588ab57773cc8d43ff4/users/66d9603539dc496491fd108f4b73adfc/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:48.944999 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc431644-4e43-458d-bce0-7580ae2fea93 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:48.969804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-2ecbf502-ab3e-42b0-9499-210a104a6cc3 None None] MFA Rules not processed for user `f11971a594014d86968c3356d11d42f7`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:48.971262 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-2ecbf502-ab3e-42b0-9499-210a104a6cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:49.035480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc431644-4e43-458d-bce0-7580ae2fea93 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:49.052670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9504527-1752-4e6b-b207-a20e723a3f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:49.055550 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9504527-1752-4e6b-b207-a20e723a3f22 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:49.055976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9504527-1752-4e6b-b207-a20e723a3f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:49.058220 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9504527-1752-4e6b-b207-a20e723a3f22 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:49.059644 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9504527-1752-4e6b-b207-a20e723a3f22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:49.063610 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9504527-1752-4e6b-b207-a20e723a3f22 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:49.086845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 906/1797] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:49.087357 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 892/1798] 10.222.0.16 () {58 vars in 1018 bytes} [Wed Aug 31 11:03:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:49.088065 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9504527-1752-4e6b-b207-a20e723a3f22 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:49.099883 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9504527-1752-4e6b-b207-a20e723a3f22 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:49.106019 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3f43a-7008-4386-b818-f13d93f488dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:49.107556 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3f43a-7008-4386-b818-f13d93f488dd admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:49.107835 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3f43a-7008-4386-b818-f13d93f488dd admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:49.108105 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3f43a-7008-4386-b818-f13d93f488dd admin admin] PATH_INFO: `/v3/projects/7d928297537a4588ab57773cc8d43ff4/users/66d9603539dc496491fd108f4b73adfc/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:49.111288 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3f43a-7008-4386-b818-f13d93f488dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d928297537a4588ab57773cc8d43ff4, user_id=66d9603539dc496491fd108f4b73adfc, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:49.113960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3f43a-7008-4386-b818-f13d93f488dd admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:49.127169 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 907/1799] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:49 2022] PUT /identity/v3/projects/7d928297537a4588ab57773cc8d43ff4/users/66d9603539dc496491fd108f4b73adfc/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:49.139044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0a40c-1a4a-4659-9a96-43e446c35c9b None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:49.139337 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0a40c-1a4a-4659-9a96-43e446c35c9b None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:49.139614 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0a40c-1a4a-4659-9a96-43e446c35c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:49.147874 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3f43a-7008-4386-b818-f13d93f488dd admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:49.150252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-f3e3f43a-7008-4386-b818-f13d93f488dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:49.187819 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-a8a0a40c-1a4a-4659-9a96-43e446c35c9b None None] MFA Rules not processed for user `66d9603539dc496491fd108f4b73adfc`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:49.191621 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a8a0a40c-1a4a-4659-9a96-43e446c35c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:49.284421 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3f43a-7008-4386-b818-f13d93f488dd admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:49.291928 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092705f-44cb-4d10-b678-d1e6d5011a9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:49.292705 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092705f-44cb-4d10-b678-d1e6d5011a9f service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:49.293199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092705f-44cb-4d10-b678-d1e6d5011a9f service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:49.293662 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092705f-44cb-4d10-b678-d1e6d5011a9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:49.305158 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d092705f-44cb-4d10-b678-d1e6d5011a9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:49.307660 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d092705f-44cb-4d10-b678-d1e6d5011a9f service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:49.319716 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 908/1800] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:49.353714 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 893/1801] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:49 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:49.356814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092705f-44cb-4d10-b678-d1e6d5011a9f service nova] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:49.434183 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092705f-44cb-4d10-b678-d1e6d5011a9f service nova] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:49.438217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab1bd3-d5d4-400f-823a-f7c53a294de2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:49.440567 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab1bd3-d5d4-400f-823a-f7c53a294de2 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:49.440930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab1bd3-d5d4-400f-823a-f7c53a294de2 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:49.441203 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab1bd3-d5d4-400f-823a-f7c53a294de2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:49.451652 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab1bd3-d5d4-400f-823a-f7c53a294de2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:49.453083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab1bd3-d5d4-400f-823a-f7c53a294de2 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:49.468329 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 909/1802] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:49 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:03:49.945632 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:50.996322 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:51.302928 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0a40c-1a4a-4659-9a96-43e446c35c9b None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:51.304336 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a8a0a40c-1a4a-4659-9a96-43e446c35c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:51.371782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0a40c-1a4a-4659-9a96-43e446c35c9b None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:51.376662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae165b2-8186-4251-8f09-a589dde41c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:51.377298 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae165b2-8186-4251-8f09-a589dde41c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:51.377576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae165b2-8186-4251-8f09-a589dde41c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:51.377853 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae165b2-8186-4251-8f09-a589dde41c6f admin admin] PATH_INFO: `/v3/users/12c648102609454aa480eb68b15dcda1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:51.379262 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae165b2-8186-4251-8f09-a589dde41c6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=12c648102609454aa480eb68b15dcda1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:51.381178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae165b2-8186-4251-8f09-a589dde41c6f admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:51.693165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-1ae165b2-8186-4251-8f09-a589dde41c6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12c648102609454aa480eb68b15dcda1'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:51.800237 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-1ae165b2-8186-4251-8f09-a589dde41c6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12c648102609454aa480eb68b15dcda1'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:51.800869 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-1ae165b2-8186-4251-8f09-a589dde41c6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12c648102609454aa480eb68b15dcda1'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:51.801478 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-1ae165b2-8186-4251-8f09-a589dde41c6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12c648102609454aa480eb68b15dcda1'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:51.814710 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-1ae165b2-8186-4251-8f09-a589dde41c6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12c648102609454aa480eb68b15dcda1'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:51.827328 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 894/1803] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:03:51 2022] DELETE /identity/v3/users/12c648102609454aa480eb68b15dcda1 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:51.832246 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f5dc2-c52b-4bdd-8cec-80d1e38d3816 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:51.832705 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f5dc2-c52b-4bdd-8cec-80d1e38d3816 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:51.832829 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f5dc2-c52b-4bdd-8cec-80d1e38d3816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:51.852432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-992f5dc2-c52b-4bdd-8cec-80d1e38d3816 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:51.854584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-992f5dc2-c52b-4bdd-8cec-80d1e38d3816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:51.913838 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 910/1804] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:51.924368 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae165b2-8186-4251-8f09-a589dde41c6f admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:51.925767 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-1ae165b2-8186-4251-8f09-a589dde41c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:51.982352 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae165b2-8186-4251-8f09-a589dde41c6f admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:51.986704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f794fd-769b-4b46-b977-b452d06fe3e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:51.993645 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f794fd-769b-4b46-b977-b452d06fe3e3 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:51.993645 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f794fd-769b-4b46-b977-b452d06fe3e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:51.993645 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f794fd-769b-4b46-b977-b452d06fe3e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:51.997943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f794fd-769b-4b46-b977-b452d06fe3e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:51.999388 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f794fd-769b-4b46-b977-b452d06fe3e3 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:52.009062 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 895/1805] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:52.511696 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f5dc2-c52b-4bdd-8cec-80d1e38d3816 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:52.513426 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-992f5dc2-c52b-4bdd-8cec-80d1e38d3816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:52.532463 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f5dc2-c52b-4bdd-8cec-80d1e38d3816 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:52.544487 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a9151-f250-481c-bf71-9e1a3e63fe89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:52.545138 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a9151-f250-481c-bf71-9e1a3e63fe89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:52.545415 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a9151-f250-481c-bf71-9e1a3e63fe89 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:52.545688 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a9151-f250-481c-bf71-9e1a3e63fe89 admin admin] PATH_INFO: `/v3/projects/e284aaf67cad42faa415a41b0097e82e` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:52.546873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a9151-f250-481c-bf71-9e1a3e63fe89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e284aaf67cad42faa415a41b0097e82e)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:52.548815 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a9151-f250-481c-bf71-9e1a3e63fe89 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:52.694155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-dc0a9151-f250-481c-bf71-9e1a3e63fe89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e284aaf67cad42faa415a41b0097e82e', 'request_id': 'req-dc0a9151-f250-481c-bf71-9e1a3e63fe89'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:52.831252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-dc0a9151-f250-481c-bf71-9e1a3e63fe89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e284aaf67cad42faa415a41b0097e82e', 'request_id': 'req-dc0a9151-f250-481c-bf71-9e1a3e63fe89'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:52.840938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-dc0a9151-f250-481c-bf71-9e1a3e63fe89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e284aaf67cad42faa415a41b0097e82e', 'request_id': 'req-dc0a9151-f250-481c-bf71-9e1a3e63fe89'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:52.856044 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-dc0a9151-f250-481c-bf71-9e1a3e63fe89 admin admin] The token cache is being invalidate because project e284aaf67cad42faa415a41b0097e82e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:03:52.856338 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-dc0a9151-f250-481c-bf71-9e1a3e63fe89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:52.856994 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-dc0a9151-f250-481c-bf71-9e1a3e63fe89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:52.858661 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 911/1806] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:03:52 2022] DELETE /identity/v3/projects/e284aaf67cad42faa415a41b0097e82e => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:52.862690 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f794fd-769b-4b46-b977-b452d06fe3e3 service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:52.872764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f794fd-769b-4b46-b977-b452d06fe3e3 service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:52.929941 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167aef54-ff91-4ea9-a9b5-5b67aa633530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:52.930788 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167aef54-ff91-4ea9-a9b5-5b67aa633530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:52.930997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167aef54-ff91-4ea9-a9b5-5b67aa633530 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:52.931379 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167aef54-ff91-4ea9-a9b5-5b67aa633530 admin admin] PATH_INFO: `/v3/users/fd5f32a0749c47f49568d2b7d25217ad` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:52.932537 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167aef54-ff91-4ea9-a9b5-5b67aa633530 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd5f32a0749c47f49568d2b7d25217ad)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:52.934517 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167aef54-ff91-4ea9-a9b5-5b67aa633530 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:53.157326 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-167aef54-ff91-4ea9-a9b5-5b67aa633530 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd5f32a0749c47f49568d2b7d25217ad'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:53.231278 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-167aef54-ff91-4ea9-a9b5-5b67aa633530 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd5f32a0749c47f49568d2b7d25217ad'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:53.232761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:53.233440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-167aef54-ff91-4ea9-a9b5-5b67aa633530 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd5f32a0749c47f49568d2b7d25217ad'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:53.234167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-167aef54-ff91-4ea9-a9b5-5b67aa633530 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd5f32a0749c47f49568d2b7d25217ad'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:53.247443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-167aef54-ff91-4ea9-a9b5-5b67aa633530 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd5f32a0749c47f49568d2b7d25217ad'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:53.260847 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 896/1807] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:03:52 2022] DELETE /identity/v3/users/fd5f32a0749c47f49568d2b7d25217ad => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:53.477169 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:53.866242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a9151-f250-481c-bf71-9e1a3e63fe89 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:53.866242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-dc0a9151-f250-481c-bf71-9e1a3e63fe89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:53.879121 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a9151-f250-481c-bf71-9e1a3e63fe89 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:53.964751 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b262f-f18c-4dd0-8f7b-21f2b53800ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:53.965426 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b262f-f18c-4dd0-8f7b-21f2b53800ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:53.965730 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b262f-f18c-4dd0-8f7b-21f2b53800ec admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:53.965986 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b262f-f18c-4dd0-8f7b-21f2b53800ec admin admin] PATH_INFO: `/v3/projects/1ab100be58bf4896a0c8c04befc9a072` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:53.967378 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980b262f-f18c-4dd0-8f7b-21f2b53800ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ab100be58bf4896a0c8c04befc9a072)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:53.969397 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980b262f-f18c-4dd0-8f7b-21f2b53800ec admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:54.084599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-980b262f-f18c-4dd0-8f7b-21f2b53800ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ab100be58bf4896a0c8c04befc9a072', 'request_id': 'req-980b262f-f18c-4dd0-8f7b-21f2b53800ec'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:54.238945 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167aef54-ff91-4ea9-a9b5-5b67aa633530 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:54.241369 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-167aef54-ff91-4ea9-a9b5-5b67aa633530 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:54.257783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167aef54-ff91-4ea9-a9b5-5b67aa633530 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:54.271116 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-980b262f-f18c-4dd0-8f7b-21f2b53800ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ab100be58bf4896a0c8c04befc9a072', 'request_id': 'req-980b262f-f18c-4dd0-8f7b-21f2b53800ec'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:54.278496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-980b262f-f18c-4dd0-8f7b-21f2b53800ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ab100be58bf4896a0c8c04befc9a072', 'request_id': 'req-980b262f-f18c-4dd0-8f7b-21f2b53800ec'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:54.290993 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-980b262f-f18c-4dd0-8f7b-21f2b53800ec admin admin] The token cache is being invalidate because project 1ab100be58bf4896a0c8c04befc9a072 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:03:54.291809 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-980b262f-f18c-4dd0-8f7b-21f2b53800ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:54.292646 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-980b262f-f18c-4dd0-8f7b-21f2b53800ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:54.294180 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 912/1808] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:03:53 2022] DELETE /identity/v3/projects/1ab100be58bf4896a0c8c04befc9a072 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:54.303697 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be594be4-685f-4ce6-8d26-207702441933 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:54.303911 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be594be4-685f-4ce6-8d26-207702441933 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:54.304155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be594be4-685f-4ce6-8d26-207702441933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:54.327858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2881c-31cd-4187-a64b-3f54ba540c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:54.328100 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-be594be4-685f-4ce6-8d26-207702441933 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:54.329180 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2881c-31cd-4187-a64b-3f54ba540c47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:54.329296 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-be594be4-685f-4ce6-8d26-207702441933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:54.329764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2881c-31cd-4187-a64b-3f54ba540c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:54.330261 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2881c-31cd-4187-a64b-3f54ba540c47 admin admin] PATH_INFO: `/v3/users/542aa8bea90541f1b8e54bbfe423a392` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:54.332422 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b2881c-31cd-4187-a64b-3f54ba540c47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=542aa8bea90541f1b8e54bbfe423a392)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:54.334779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b2881c-31cd-4187-a64b-3f54ba540c47 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:54.366596 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 913/1809] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:54.368080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0900bd-a693-428e-856f-d558fe2ba388 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:54.368444 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0900bd-a693-428e-856f-d558fe2ba388 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:54.368690 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0900bd-a693-428e-856f-d558fe2ba388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:54.401268 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-4c0900bd-a693-428e-856f-d558fe2ba388 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:54.402601 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-4c0900bd-a693-428e-856f-d558fe2ba388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:54.411525 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 914/1810] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:54.423639 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0900bd-a693-428e-856f-d558fe2ba388 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:54.435247 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0900bd-a693-428e-856f-d558fe2ba388 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:54.446039 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd842865-b434-4823-a270-24da62e3008d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:54.446891 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd842865-b434-4823-a270-24da62e3008d admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:54.447292 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd842865-b434-4823-a270-24da62e3008d admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:54.447782 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd842865-b434-4823-a270-24da62e3008d admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:54.448601 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd842865-b434-4823-a270-24da62e3008d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:03:54.448999 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd842865-b434-4823-a270-24da62e3008d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:54.450759 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd842865-b434-4823-a270-24da62e3008d admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:54.469916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 915/1811] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:03:54.476040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd842865-b434-4823-a270-24da62e3008d admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:54.495866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd842865-b434-4823-a270-24da62e3008d admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:54.504289 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ca6fb-8ac6-45d0-8c63-045b9ec8a679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:54.505228 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ca6fb-8ac6-45d0-8c63-045b9ec8a679 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:54.505711 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ca6fb-8ac6-45d0-8c63-045b9ec8a679 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:54.506213 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ca6fb-8ac6-45d0-8c63-045b9ec8a679 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:54.507310 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927ca6fb-8ac6-45d0-8c63-045b9ec8a679 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:54.511311 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927ca6fb-8ac6-45d0-8c63-045b9ec8a679 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:54.719120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 916/1812] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:03:54 2022] POST /identity/v3/projects => generated 375 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:54.726833 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ca6fb-8ac6-45d0-8c63-045b9ec8a679 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:54.791871 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ca6fb-8ac6-45d0-8c63-045b9ec8a679 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:54.793545 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-a5b2881c-31cd-4187-a64b-3f54ba540c47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '542aa8bea90541f1b8e54bbfe423a392'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:54.800137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b08b5-4229-4630-8959-eec349d6bdc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:54.800791 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b08b5-4229-4630-8959-eec349d6bdc5 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:54.801073 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b08b5-4229-4630-8959-eec349d6bdc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:54.801346 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b08b5-4229-4630-8959-eec349d6bdc5 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:54.801993 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b08b5-4229-4630-8959-eec349d6bdc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:54.804021 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b08b5-4229-4630-8959-eec349d6bdc5 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:54.858618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-a5b2881c-31cd-4187-a64b-3f54ba540c47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '542aa8bea90541f1b8e54bbfe423a392'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:54.859662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-a5b2881c-31cd-4187-a64b-3f54ba540c47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '542aa8bea90541f1b8e54bbfe423a392'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:54.860452 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-a5b2881c-31cd-4187-a64b-3f54ba540c47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '542aa8bea90541f1b8e54bbfe423a392'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:54.875748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-a5b2881c-31cd-4187-a64b-3f54ba540c47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '542aa8bea90541f1b8e54bbfe423a392'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:54.888801 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 897/1813] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:03:54 2022] DELETE /identity/v3/users/542aa8bea90541f1b8e54bbfe423a392 => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:54.893481 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a7869-0eca-45f6-8f5c-8cb29480af00 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:54.895169 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a7869-0eca-45f6-8f5c-8cb29480af00 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:54.895169 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a7869-0eca-45f6-8f5c-8cb29480af00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:54.916901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-c02a7869-0eca-45f6-8f5c-8cb29480af00 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:54.918420 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-c02a7869-0eca-45f6-8f5c-8cb29480af00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:54.924485 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 917/1814] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:03:54 2022] POST /identity/v3/users => generated 288 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:03:54.930810 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b08b5-4229-4630-8959-eec349d6bdc5 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:54.935450 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-106b08b5-4229-4630-8959-eec349d6bdc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:54.967083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b08b5-4229-4630-8959-eec349d6bdc5 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:55.035919 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 898/1815] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:03:55.044489 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a7869-0eca-45f6-8f5c-8cb29480af00 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:55.046058 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-c02a7869-0eca-45f6-8f5c-8cb29480af00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:55.091156 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d6a7c5-db4a-46bf-961a-1e1c6750c13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:55.091794 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6a7c5-db4a-46bf-961a-1e1c6750c13a admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:55.092380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6a7c5-db4a-46bf-961a-1e1c6750c13a admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:55.092705 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6a7c5-db4a-46bf-961a-1e1c6750c13a admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:55.093568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d6a7c5-db4a-46bf-961a-1e1c6750c13a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:55.095499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d6a7c5-db4a-46bf-961a-1e1c6750c13a admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:55.121328 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 918/1816] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:55.136601 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d6a7c5-db4a-46bf-961a-1e1c6750c13a admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:55.144032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a7869-0eca-45f6-8f5c-8cb29480af00 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:55.148849 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa079df2-5cec-44ba-a65f-ac16d9a7774f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:55.149431 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d6a7c5-db4a-46bf-961a-1e1c6750c13a admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:55.151919 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa079df2-5cec-44ba-a65f-ac16d9a7774f service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:55.151919 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa079df2-5cec-44ba-a65f-ac16d9a7774f service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:55.153045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa079df2-5cec-44ba-a65f-ac16d9a7774f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:55.165428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa079df2-5cec-44ba-a65f-ac16d9a7774f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:55.167710 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa079df2-5cec-44ba-a65f-ac16d9a7774f service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:55.176413 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2df988-55ad-4a3f-8aed-e827c8b69fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:55.180735 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2df988-55ad-4a3f-8aed-e827c8b69fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:55.180735 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2df988-55ad-4a3f-8aed-e827c8b69fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:55.180735 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2df988-55ad-4a3f-8aed-e827c8b69fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:55.180735 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2df988-55ad-4a3f-8aed-e827c8b69fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:55.180735 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2df988-55ad-4a3f-8aed-e827c8b69fc1 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:55.188828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 899/1817] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:03:55.219489 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 919/1818] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:55.223717 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa079df2-5cec-44ba-a65f-ac16d9a7774f service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:55.238152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa079df2-5cec-44ba-a65f-ac16d9a7774f service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:55.246927 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24515f04-5889-4160-9a7d-e7d3ce24ad99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:55.247746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24515f04-5889-4160-9a7d-e7d3ce24ad99 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:55.248119 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24515f04-5889-4160-9a7d-e7d3ce24ad99 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:55.248486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24515f04-5889-4160-9a7d-e7d3ce24ad99 admin admin] PATH_INFO: `/v3/projects/3088580f9e9149849dd00e609cee8f18/users/e1f8b7d5f65b4d53bdf3077f80206496/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:55.271492 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24515f04-5889-4160-9a7d-e7d3ce24ad99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3088580f9e9149849dd00e609cee8f18, user_id=e1f8b7d5f65b4d53bdf3077f80206496, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:55.292623 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24515f04-5889-4160-9a7d-e7d3ce24ad99 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:55.615529 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2df988-55ad-4a3f-8aed-e827c8b69fc1 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:55.616938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-fd2df988-55ad-4a3f-8aed-e827c8b69fc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:55.650722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 900/1819] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:55 2022] PUT /identity/v3/projects/3088580f9e9149849dd00e609cee8f18/users/e1f8b7d5f65b4d53bdf3077f80206496/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:55.662266 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70694fa7-f6d7-452c-9ee4-f7fbdd0c99d5 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:55.662666 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70694fa7-f6d7-452c-9ee4-f7fbdd0c99d5 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:55.663010 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70694fa7-f6d7-452c-9ee4-f7fbdd0c99d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:55.704650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-70694fa7-f6d7-452c-9ee4-f7fbdd0c99d5 None None] MFA Rules not processed for user `e1f8b7d5f65b4d53bdf3077f80206496`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:55.705881 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-70694fa7-f6d7-452c-9ee4-f7fbdd0c99d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:55.721953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2df988-55ad-4a3f-8aed-e827c8b69fc1 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:55.832507 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 901/1820] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:03:55.837353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70694fa7-f6d7-452c-9ee4-f7fbdd0c99d5 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:55.845403 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92189e-459e-48dd-b5a5-ae16e5c10fd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:55.846086 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92189e-459e-48dd-b5a5-ae16e5c10fd2 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:55.846898 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92189e-459e-48dd-b5a5-ae16e5c10fd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:55.854073 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92189e-459e-48dd-b5a5-ae16e5c10fd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:55.854073 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2b92189e-459e-48dd-b5a5-ae16e5c10fd2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:55.976372 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92189e-459e-48dd-b5a5-ae16e5c10fd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:55.979199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92189e-459e-48dd-b5a5-ae16e5c10fd2 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:55.990436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70694fa7-f6d7-452c-9ee4-f7fbdd0c99d5 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:55.993578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 920/1821] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:55 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:56.010807 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e07922-8d15-4357-8968-53458d67705c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:56.012331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e07922-8d15-4357-8968-53458d67705c admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:56.012687 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e07922-8d15-4357-8968-53458d67705c admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:56.013049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e07922-8d15-4357-8968-53458d67705c admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:56.013777 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e07922-8d15-4357-8968-53458d67705c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:56.019030 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e07922-8d15-4357-8968-53458d67705c admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:56.269976 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 902/1822] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:03:55 2022] POST /identity/v3/projects => generated 377 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:56.274321 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92189e-459e-48dd-b5a5-ae16e5c10fd2 service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:56.296696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e07922-8d15-4357-8968-53458d67705c admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:56.305896 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92189e-459e-48dd-b5a5-ae16e5c10fd2 service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:56.307908 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-15e07922-8d15-4357-8968-53458d67705c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:56.474168 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed65fb7-887d-443c-9143-37eabce5736d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:56.480549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed65fb7-887d-443c-9143-37eabce5736d admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:56.480855 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed65fb7-887d-443c-9143-37eabce5736d admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:56.481127 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed65fb7-887d-443c-9143-37eabce5736d admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:56.481818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed65fb7-887d-443c-9143-37eabce5736d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:56.486542 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed65fb7-887d-443c-9143-37eabce5736d admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:56.499100 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e07922-8d15-4357-8968-53458d67705c admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:56.508127 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4123403-00c5-4e5b-8f07-cc1456deedd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:56.511097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4123403-00c5-4e5b-8f07-cc1456deedd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:56.511097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4123403-00c5-4e5b-8f07-cc1456deedd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:56.511097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4123403-00c5-4e5b-8f07-cc1456deedd6 admin admin] PATH_INFO: `/v3/projects/d48032da9f7f43a1954f462c2da1d408` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:56.511339 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4123403-00c5-4e5b-8f07-cc1456deedd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d48032da9f7f43a1954f462c2da1d408)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:56.513375 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4123403-00c5-4e5b-8f07-cc1456deedd6 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:56.835846 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 921/1823] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:03:56 2022] POST /identity/v3/users => generated 289 bytes in 562 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:03:56.840014 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed65fb7-887d-443c-9143-37eabce5736d admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:56.859558 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed65fb7-887d-443c-9143-37eabce5736d admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:56.872706 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49043f45-fed8-4b2c-bc64-59d0a457a520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:56.877779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49043f45-fed8-4b2c-bc64-59d0a457a520 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:56.877779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49043f45-fed8-4b2c-bc64-59d0a457a520 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:56.877779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49043f45-fed8-4b2c-bc64-59d0a457a520 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:56.877779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49043f45-fed8-4b2c-bc64-59d0a457a520 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:56.878319 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49043f45-fed8-4b2c-bc64-59d0a457a520 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:56.900873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 922/1824] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:03:56.904774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49043f45-fed8-4b2c-bc64-59d0a457a520 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:56.923817 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49043f45-fed8-4b2c-bc64-59d0a457a520 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:56.965873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-c4123403-00c5-4e5b-8f07-cc1456deedd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd48032da9f7f43a1954f462c2da1d408', 'request_id': 'req-c4123403-00c5-4e5b-8f07-cc1456deedd6'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:56.985116 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-c4123403-00c5-4e5b-8f07-cc1456deedd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd48032da9f7f43a1954f462c2da1d408', 'request_id': 'req-c4123403-00c5-4e5b-8f07-cc1456deedd6'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:56.997933 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-c4123403-00c5-4e5b-8f07-cc1456deedd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd48032da9f7f43a1954f462c2da1d408', 'request_id': 'req-c4123403-00c5-4e5b-8f07-cc1456deedd6'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:57.033589 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177d7dd-dd57-4aba-9f46-03bdc0e03aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:57.035226 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177d7dd-dd57-4aba-9f46-03bdc0e03aae admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:57.035840 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177d7dd-dd57-4aba-9f46-03bdc0e03aae admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:57.036317 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177d7dd-dd57-4aba-9f46-03bdc0e03aae admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:57.037347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177d7dd-dd57-4aba-9f46-03bdc0e03aae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:57.039513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177d7dd-dd57-4aba-9f46-03bdc0e03aae admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:57.055840 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 923/1825] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:03:57.062651 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177d7dd-dd57-4aba-9f46-03bdc0e03aae admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:57.083717 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177d7dd-dd57-4aba-9f46-03bdc0e03aae admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:57.092794 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a17912-a89a-4728-b88e-1985dbda18a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:57.093504 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a17912-a89a-4728-b88e-1985dbda18a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:57.099490 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a17912-a89a-4728-b88e-1985dbda18a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:57.099790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a17912-a89a-4728-b88e-1985dbda18a4 admin admin] PATH_INFO: `/v3/projects/bc5978f3eb14436bac4c2617092de5a6/users/01b5c25be2774354b691275d25bdd840/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:57.120143 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a17912-a89a-4728-b88e-1985dbda18a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc5978f3eb14436bac4c2617092de5a6, user_id=01b5c25be2774354b691275d25bdd840, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:57.126434 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a17912-a89a-4728-b88e-1985dbda18a4 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:57.157737 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-c4123403-00c5-4e5b-8f07-cc1456deedd6 admin admin] The token cache is being invalidate because project d48032da9f7f43a1954f462c2da1d408 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:03:57.158043 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-c4123403-00c5-4e5b-8f07-cc1456deedd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:57.158774 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-c4123403-00c5-4e5b-8f07-cc1456deedd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:57.160622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 903/1826] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:03:56 2022] DELETE /identity/v3/projects/d48032da9f7f43a1954f462c2da1d408 => generated 0 bytes in 865 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:57.164648 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4123403-00c5-4e5b-8f07-cc1456deedd6 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:57.166796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-c4123403-00c5-4e5b-8f07-cc1456deedd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:57.186052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4123403-00c5-4e5b-8f07-cc1456deedd6 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:57.196798 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21505f60-0963-467e-8842-509b3c8e86db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:57.197671 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21505f60-0963-467e-8842-509b3c8e86db admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:57.198022 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21505f60-0963-467e-8842-509b3c8e86db admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:57.198360 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21505f60-0963-467e-8842-509b3c8e86db admin admin] PATH_INFO: `/v3/users/66d9603539dc496491fd108f4b73adfc` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:57.204711 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21505f60-0963-467e-8842-509b3c8e86db admin admin] RBAC: Authorizing `identity:delete_user(user_id=66d9603539dc496491fd108f4b73adfc)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:57.210929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21505f60-0963-467e-8842-509b3c8e86db admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:57.219027 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 924/1827] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:57 2022] PUT /identity/v3/projects/bc5978f3eb14436bac4c2617092de5a6/users/01b5c25be2774354b691275d25bdd840/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:57.230658 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c183f-2de1-439b-8687-82c00434f293 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:57.231255 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c183f-2de1-439b-8687-82c00434f293 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:57.231769 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c183f-2de1-439b-8687-82c00434f293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:57.273064 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-2e4c183f-2de1-439b-8687-82c00434f293 None None] MFA Rules not processed for user `01b5c25be2774354b691275d25bdd840`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:57.283722 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2e4c183f-2de1-439b-8687-82c00434f293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:57.350093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 925/1828] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:03:57.506340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c183f-2de1-439b-8687-82c00434f293 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:57.506340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2e4c183f-2de1-439b-8687-82c00434f293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:57.506340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c183f-2de1-439b-8687-82c00434f293 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:57.507718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c5dbe-29b2-4190-b490-63fa1d2823cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:57.507718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c5dbe-29b2-4190-b490-63fa1d2823cd admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:57.507718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c5dbe-29b2-4190-b490-63fa1d2823cd admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:57.507718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c5dbe-29b2-4190-b490-63fa1d2823cd admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:57.508293 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c5dbe-29b2-4190-b490-63fa1d2823cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:57.508293 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c5dbe-29b2-4190-b490-63fa1d2823cd admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:57.788277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 926/1829] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:03:57 2022] POST /identity/v3/projects => generated 375 bytes in 430 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:03:57.789285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c5dbe-29b2-4190-b490-63fa1d2823cd admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:57.863603 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-21505f60-0963-467e-8842-509b3c8e86db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66d9603539dc496491fd108f4b73adfc'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:57.901178 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c5dbe-29b2-4190-b490-63fa1d2823cd admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:57.901899 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03468e45-cc24-4435-9307-ef451c2265cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:57.902645 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03468e45-cc24-4435-9307-ef451c2265cb admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:57.902970 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03468e45-cc24-4435-9307-ef451c2265cb admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:57.903388 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03468e45-cc24-4435-9307-ef451c2265cb admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:57.904084 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03468e45-cc24-4435-9307-ef451c2265cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:57.906317 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03468e45-cc24-4435-9307-ef451c2265cb admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:58.139332 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-21505f60-0963-467e-8842-509b3c8e86db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66d9603539dc496491fd108f4b73adfc'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:58.140225 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-21505f60-0963-467e-8842-509b3c8e86db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66d9603539dc496491fd108f4b73adfc'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:58.141039 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-21505f60-0963-467e-8842-509b3c8e86db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66d9603539dc496491fd108f4b73adfc'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:58.151052 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 927/1830] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:03:57 2022] POST /identity/v3/users => generated 294 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:03:58.154469 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03468e45-cc24-4435-9307-ef451c2265cb admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:58.155760 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-03468e45-cc24-4435-9307-ef451c2265cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:58.170653 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03468e45-cc24-4435-9307-ef451c2265cb admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:58.174379 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-21505f60-0963-467e-8842-509b3c8e86db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66d9603539dc496491fd108f4b73adfc'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:58.176143 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925c79f-c8a9-432d-88aa-c6c8474ce399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:58.176945 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925c79f-c8a9-432d-88aa-c6c8474ce399 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:58.177630 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925c79f-c8a9-432d-88aa-c6c8474ce399 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:58.177954 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925c79f-c8a9-432d-88aa-c6c8474ce399 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:58.179033 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2925c79f-c8a9-432d-88aa-c6c8474ce399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:58.182456 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2925c79f-c8a9-432d-88aa-c6c8474ce399 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:58.198441 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 928/1831] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:03:58.201984 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925c79f-c8a9-432d-88aa-c6c8474ce399 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:58.219767 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 904/1832] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:03:57 2022] DELETE /identity/v3/users/66d9603539dc496491fd108f4b73adfc => generated 0 bytes in 1056 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:03:58.230459 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925c79f-c8a9-432d-88aa-c6c8474ce399 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:58.324139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211499a1-3f9f-473e-b94b-dae7d2475ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:58.325709 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211499a1-3f9f-473e-b94b-dae7d2475ac2 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:58.326206 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211499a1-3f9f-473e-b94b-dae7d2475ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:58.326826 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211499a1-3f9f-473e-b94b-dae7d2475ac2 admin admin] PATH_INFO: `/v3/projects/c8c29cff9bf245ee8c5709311ee06dec/users/528022f1b6e143b6933cfdcc874772c7/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:58.358110 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211499a1-3f9f-473e-b94b-dae7d2475ac2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8c29cff9bf245ee8c5709311ee06dec, user_id=528022f1b6e143b6933cfdcc874772c7, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:58.361575 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211499a1-3f9f-473e-b94b-dae7d2475ac2 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:58.693848 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 929/1833] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:58 2022] PUT /identity/v3/projects/c8c29cff9bf245ee8c5709311ee06dec/users/528022f1b6e143b6933cfdcc874772c7/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:03:58.694917 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21505f60-0963-467e-8842-509b3c8e86db admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:58.707432 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21505f60-0963-467e-8842-509b3c8e86db admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:58.815981 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d5598-9a24-4168-9eb1-e4968fbb7db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:58.816850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d5598-9a24-4168-9eb1-e4968fbb7db6 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:58.817353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d5598-9a24-4168-9eb1-e4968fbb7db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:58.817688 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d5598-9a24-4168-9eb1-e4968fbb7db6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:58.818545 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d5598-9a24-4168-9eb1-e4968fbb7db6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:58.820495 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d5598-9a24-4168-9eb1-e4968fbb7db6 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:58.839466 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211499a1-3f9f-473e-b94b-dae7d2475ac2 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:58.848689 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 905/1834] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:03:58.856184 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211499a1-3f9f-473e-b94b-dae7d2475ac2 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:58.862274 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9aae92-ee60-495c-a85c-f38fc20fb46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:58.862994 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9aae92-ee60-495c-a85c-f38fc20fb46f admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:58.863275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9aae92-ee60-495c-a85c-f38fc20fb46f admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:58.863545 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9aae92-ee60-495c-a85c-f38fc20fb46f admin admin] PATH_INFO: `/v3/projects/c8c29cff9bf245ee8c5709311ee06dec/users/528022f1b6e143b6933cfdcc874772c7/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:58.865680 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9aae92-ee60-495c-a85c-f38fc20fb46f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8c29cff9bf245ee8c5709311ee06dec, user_id=528022f1b6e143b6933cfdcc874772c7, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:58.869349 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9aae92-ee60-495c-a85c-f38fc20fb46f admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:58.890902 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 930/1835] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:03:58 2022] PUT /identity/v3/projects/c8c29cff9bf245ee8c5709311ee06dec/users/528022f1b6e143b6933cfdcc874772c7/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:03:58.894584 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c35292-c201-4085-b9ee-871b57c6f992 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:58.897358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c35292-c201-4085-b9ee-871b57c6f992 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:58.897358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c35292-c201-4085-b9ee-871b57c6f992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:58.940355 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-81c35292-c201-4085-b9ee-871b57c6f992 None None] MFA Rules not processed for user `528022f1b6e143b6933cfdcc874772c7`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:58.943094 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-81c35292-c201-4085-b9ee-871b57c6f992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:59.034345 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 906/1836] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:03:58 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:59.079124 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9aae92-ee60-495c-a85c-f38fc20fb46f admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:59.081365 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-ee9aae92-ee60-495c-a85c-f38fc20fb46f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:59.219864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9aae92-ee60-495c-a85c-f38fc20fb46f admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:59.225603 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f883d-abfc-4627-bcb6-7956f9c4cec4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:59.226600 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f883d-abfc-4627-bcb6-7956f9c4cec4 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:59.226880 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f883d-abfc-4627-bcb6-7956f9c4cec4 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:59.227228 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f883d-abfc-4627-bcb6-7956f9c4cec4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:59.232683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-6a9f883d-abfc-4627-bcb6-7956f9c4cec4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:59.319688 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9f883d-abfc-4627-bcb6-7956f9c4cec4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:59.321273 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9f883d-abfc-4627-bcb6-7956f9c4cec4 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:59.339170 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c35292-c201-4085-b9ee-871b57c6f992 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:59.341895 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-81c35292-c201-4085-b9ee-871b57c6f992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:59.342231 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 931/1837] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:03:59.457112 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:59.484377 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c35292-c201-4085-b9ee-871b57c6f992 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:59.493581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e34ab-f813-4936-b295-b1e85a6cff56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:59.495491 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e34ab-f813-4936-b295-b1e85a6cff56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:59.496062 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e34ab-f813-4936-b295-b1e85a6cff56 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:59.496567 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e34ab-f813-4936-b295-b1e85a6cff56 admin admin] PATH_INFO: `/v3/projects/7d928297537a4588ab57773cc8d43ff4` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:59.500768 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e34ab-f813-4936-b295-b1e85a6cff56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d928297537a4588ab57773cc8d43ff4)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:59.506188 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e34ab-f813-4936-b295-b1e85a6cff56 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:59.692662 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:59.826526 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-e80e34ab-f813-4936-b295-b1e85a6cff56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d928297537a4588ab57773cc8d43ff4', 'request_id': 'req-e80e34ab-f813-4936-b295-b1e85a6cff56'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:59.844289 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-e80e34ab-f813-4936-b295-b1e85a6cff56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d928297537a4588ab57773cc8d43ff4', 'request_id': 'req-e80e34ab-f813-4936-b295-b1e85a6cff56'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:03:59.852635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-e80e34ab-f813-4936-b295-b1e85a6cff56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d928297537a4588ab57773cc8d43ff4', 'request_id': 'req-e80e34ab-f813-4936-b295-b1e85a6cff56'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:00.056330 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-e80e34ab-f813-4936-b295-b1e85a6cff56 admin admin] The token cache is being invalidate because project 7d928297537a4588ab57773cc8d43ff4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:04:00.057516 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-e80e34ab-f813-4936-b295-b1e85a6cff56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:00.057796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-e80e34ab-f813-4936-b295-b1e85a6cff56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:00.063126 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 907/1838] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:03:59 2022] DELETE /identity/v3/projects/7d928297537a4588ab57773cc8d43ff4 => generated 0 bytes in 721 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:00.065190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad4690-1b24-4e5f-8633-62eeae3e6e9a None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:00.065477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad4690-1b24-4e5f-8633-62eeae3e6e9a None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:00.065757 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad4690-1b24-4e5f-8633-62eeae3e6e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:00.087063 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-d7ad4690-1b24-4e5f-8633-62eeae3e6e9a None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:00.088816 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-d7ad4690-1b24-4e5f-8633-62eeae3e6e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:00.156531 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 932/1839] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:00.165750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f7d48-0b4c-4bbe-953d-f3c90cdbb38e None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:00.166042 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f7d48-0b4c-4bbe-953d-f3c90cdbb38e None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:00.166320 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f7d48-0b4c-4bbe-953d-f3c90cdbb38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:00.186669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-af1f7d48-0b4c-4bbe-953d-f3c90cdbb38e None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:00.187868 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-af1f7d48-0b4c-4bbe-953d-f3c90cdbb38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:00.196057 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 908/1840] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:00.200967 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad4690-1b24-4e5f-8633-62eeae3e6e9a None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:00.210819 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad4690-1b24-4e5f-8633-62eeae3e6e9a None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:00.216934 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59958c5-23d5-4027-81b2-4d9e84203c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:00.217609 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59958c5-23d5-4027-81b2-4d9e84203c35 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:00.217962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59958c5-23d5-4027-81b2-4d9e84203c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:00.218248 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59958c5-23d5-4027-81b2-4d9e84203c35 admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:00.218954 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59958c5-23d5-4027-81b2-4d9e84203c35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:04:00.219253 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59958c5-23d5-4027-81b2-4d9e84203c35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:00.220845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59958c5-23d5-4027-81b2-4d9e84203c35 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:00.233661 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 933/1841] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:04:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:04:00.238029 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f7d48-0b4c-4bbe-953d-f3c90cdbb38e None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:00.247529 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f7d48-0b4c-4bbe-953d-f3c90cdbb38e None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:00.252559 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93378ac-aa40-4d1f-836e-fea5683a8933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:00.257040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93378ac-aa40-4d1f-836e-fea5683a8933 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:00.257040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93378ac-aa40-4d1f-836e-fea5683a8933 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:00.257040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93378ac-aa40-4d1f-836e-fea5683a8933 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:00.257040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93378ac-aa40-4d1f-836e-fea5683a8933 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:00.257040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93378ac-aa40-4d1f-836e-fea5683a8933 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:00.413402 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 909/1842] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:04:00 2022] POST /identity/v3/projects => generated 381 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:04:00.414964 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59958c5-23d5-4027-81b2-4d9e84203c35 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:00.425014 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59958c5-23d5-4027-81b2-4d9e84203c35 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:00.484305 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9284d40e-ef36-470e-bc3f-2ceb93c268c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:00.484942 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284d40e-ef36-470e-bc3f-2ceb93c268c5 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:00.485226 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284d40e-ef36-470e-bc3f-2ceb93c268c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:00.485539 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284d40e-ef36-470e-bc3f-2ceb93c268c5 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:00.486146 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9284d40e-ef36-470e-bc3f-2ceb93c268c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:00.488136 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9284d40e-ef36-470e-bc3f-2ceb93c268c5 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:00.609261 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 934/1843] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:04:00 2022] POST /identity/v3/users => generated 291 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:04:00.613085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93378ac-aa40-4d1f-836e-fea5683a8933 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:00.624080 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93378ac-aa40-4d1f-836e-fea5683a8933 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:00.628782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d98db-1243-4dea-a0d2-4ba7bbf5c43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:00.629358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d98db-1243-4dea-a0d2-4ba7bbf5c43b admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:00.629636 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d98db-1243-4dea-a0d2-4ba7bbf5c43b admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:00.629902 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d98db-1243-4dea-a0d2-4ba7bbf5c43b admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:00.630587 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1d98db-1243-4dea-a0d2-4ba7bbf5c43b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:00.632159 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1d98db-1243-4dea-a0d2-4ba7bbf5c43b admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:00.642917 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 910/1844] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:04:00.647427 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9284d40e-ef36-470e-bc3f-2ceb93c268c5 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:00.658369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9284d40e-ef36-470e-bc3f-2ceb93c268c5 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:00.663286 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7290166-0e9d-454f-95ba-bd8cbfb8ecdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:00.663929 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7290166-0e9d-454f-95ba-bd8cbfb8ecdf admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:00.664209 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7290166-0e9d-454f-95ba-bd8cbfb8ecdf admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:00.664616 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7290166-0e9d-454f-95ba-bd8cbfb8ecdf admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:00.665291 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7290166-0e9d-454f-95ba-bd8cbfb8ecdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:00.667735 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7290166-0e9d-454f-95ba-bd8cbfb8ecdf admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:00.679199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 935/1845] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:04:00.683667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d98db-1243-4dea-a0d2-4ba7bbf5c43b admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:00.694063 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d98db-1243-4dea-a0d2-4ba7bbf5c43b admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:00.699546 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37117a29-936b-4aa9-b286-22183f0295cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:00.700169 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37117a29-936b-4aa9-b286-22183f0295cb admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:00.700448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37117a29-936b-4aa9-b286-22183f0295cb admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:00.700717 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37117a29-936b-4aa9-b286-22183f0295cb admin admin] PATH_INFO: `/v3/projects/7d94de874a584fd19a55445674fdd8c5/users/425bfdd105d0454d8e8111d88ed62f5e/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:00.716739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37117a29-936b-4aa9-b286-22183f0295cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d94de874a584fd19a55445674fdd8c5, user_id=425bfdd105d0454d8e8111d88ed62f5e, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:00.719490 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37117a29-936b-4aa9-b286-22183f0295cb admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:00.797638 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 911/1846] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:04:00 2022] PUT /identity/v3/projects/7d94de874a584fd19a55445674fdd8c5/users/425bfdd105d0454d8e8111d88ed62f5e/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:04:00.809026 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45b49b-127c-410c-bece-b585c5ba4317 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:00.809361 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45b49b-127c-410c-bece-b585c5ba4317 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:00.809587 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45b49b-127c-410c-bece-b585c5ba4317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:00.838518 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-cd45b49b-127c-410c-bece-b585c5ba4317 None None] MFA Rules not processed for user `425bfdd105d0454d8e8111d88ed62f5e`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:00.840029 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-cd45b49b-127c-410c-bece-b585c5ba4317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:00.907701 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 936/1847] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:00.912037 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37117a29-936b-4aa9-b286-22183f0295cb admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:00.987556 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37117a29-936b-4aa9-b286-22183f0295cb admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:00.992913 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec81ce-4099-473f-a580-4583af159f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:00.993532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec81ce-4099-473f-a580-4583af159f40 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:00.993817 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec81ce-4099-473f-a580-4583af159f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:00.994168 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec81ce-4099-473f-a580-4583af159f40 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:00.994953 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec81ce-4099-473f-a580-4583af159f40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:00.996855 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec81ce-4099-473f-a580-4583af159f40 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:01.107563 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 912/1848] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:04:00 2022] POST /identity/v3/projects => generated 381 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:04:01.111678 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45b49b-127c-410c-bece-b585c5ba4317 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:01.123204 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45b49b-127c-410c-bece-b585c5ba4317 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:01.179756 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad61e4-44a7-43f6-8051-e4104e089050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:01.180358 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad61e4-44a7-43f6-8051-e4104e089050 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:01.180638 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad61e4-44a7-43f6-8051-e4104e089050 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:01.180910 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad61e4-44a7-43f6-8051-e4104e089050 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:01.181666 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cad61e4-44a7-43f6-8051-e4104e089050 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:01.183990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cad61e4-44a7-43f6-8051-e4104e089050 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:01.265221 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 937/1849] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:04:01 2022] POST /identity/v3/users => generated 297 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:04:01.269088 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:01.269324 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec81ce-4099-473f-a580-4583af159f40 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:01.280111 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec81ce-4099-473f-a580-4583af159f40 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:01.284956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c28d7-e5d3-45d4-aa6d-4a722efdd876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:01.285564 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c28d7-e5d3-45d4-aa6d-4a722efdd876 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:01.285868 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c28d7-e5d3-45d4-aa6d-4a722efdd876 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:01.286143 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c28d7-e5d3-45d4-aa6d-4a722efdd876 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:01.286861 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60c28d7-e5d3-45d4-aa6d-4a722efdd876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:01.288458 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60c28d7-e5d3-45d4-aa6d-4a722efdd876 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:01.299449 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 913/1850] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:04:01.308213 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad61e4-44a7-43f6-8051-e4104e089050 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:01.322085 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad61e4-44a7-43f6-8051-e4104e089050 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:01.327593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb2eb6-ac51-48ba-913c-bd6f08b06bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:01.328297 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb2eb6-ac51-48ba-913c-bd6f08b06bd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:01.328563 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb2eb6-ac51-48ba-913c-bd6f08b06bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:01.328836 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb2eb6-ac51-48ba-913c-bd6f08b06bd1 admin admin] PATH_INFO: `/v3/projects/c0d29c92b6644689b7ff6fc578c98576/users/1a12c984891b4d19bd61a77b2e4f593d/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:01.344128 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb2eb6-ac51-48ba-913c-bd6f08b06bd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0d29c92b6644689b7ff6fc578c98576, user_id=1a12c984891b4d19bd61a77b2e4f593d, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:01.346850 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb2eb6-ac51-48ba-913c-bd6f08b06bd1 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:01.449007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 938/1851] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:04:01 2022] PUT /identity/v3/projects/c0d29c92b6644689b7ff6fc578c98576/users/1a12c984891b4d19bd61a77b2e4f593d/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:01.453534 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c28d7-e5d3-45d4-aa6d-4a722efdd876 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:01.465027 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c28d7-e5d3-45d4-aa6d-4a722efdd876 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:01.526511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8bbfa-e36e-4f1c-b751-5120c26d6b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:01.527182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8bbfa-e36e-4f1c-b751-5120c26d6b73 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:01.527459 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8bbfa-e36e-4f1c-b751-5120c26d6b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:01.527729 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8bbfa-e36e-4f1c-b751-5120c26d6b73 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:01.528367 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8bbfa-e36e-4f1c-b751-5120c26d6b73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:01.529960 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8bbfa-e36e-4f1c-b751-5120c26d6b73 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:01.540081 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 914/1852] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:04:01.544679 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb2eb6-ac51-48ba-913c-bd6f08b06bd1 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:01.555259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb2eb6-ac51-48ba-913c-bd6f08b06bd1 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:01.557627 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:01.560577 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b10ba-1d6b-4712-ae1d-d338d31684b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:01.561534 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b10ba-1d6b-4712-ae1d-d338d31684b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:01.561534 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b10ba-1d6b-4712-ae1d-d338d31684b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:01.561789 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b10ba-1d6b-4712-ae1d-d338d31684b3 admin admin] PATH_INFO: `/v3/projects/c0d29c92b6644689b7ff6fc578c98576/users/1a12c984891b4d19bd61a77b2e4f593d/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:01.563755 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b10ba-1d6b-4712-ae1d-d338d31684b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0d29c92b6644689b7ff6fc578c98576, user_id=1a12c984891b4d19bd61a77b2e4f593d, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:01.566463 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b10ba-1d6b-4712-ae1d-d338d31684b3 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:01.579358 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 939/1853] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:04:01 2022] PUT /identity/v3/projects/c0d29c92b6644689b7ff6fc578c98576/users/1a12c984891b4d19bd61a77b2e4f593d/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:01.589290 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2b03f-1231-453b-b9bb-d0250f7317c9 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:01.589576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2b03f-1231-453b-b9bb-d0250f7317c9 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:01.589854 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2b03f-1231-453b-b9bb-d0250f7317c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:01.628290 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-4ca2b03f-1231-453b-b9bb-d0250f7317c9 None None] MFA Rules not processed for user `1a12c984891b4d19bd61a77b2e4f593d`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:01.634531 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-4ca2b03f-1231-453b-b9bb-d0250f7317c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:01.699672 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 915/1854] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:01 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:01.712239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b10ba-1d6b-4712-ae1d-d338d31684b3 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:01.714155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-9e5b10ba-1d6b-4712-ae1d-d338d31684b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:01.775494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b10ba-1d6b-4712-ae1d-d338d31684b3 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:01.779618 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea4c7f-90ec-4602-bca1-c63049897773 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:01.780294 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea4c7f-90ec-4602-bca1-c63049897773 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:01.780619 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea4c7f-90ec-4602-bca1-c63049897773 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:01.780973 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea4c7f-90ec-4602-bca1-c63049897773 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:01.792582 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea4c7f-90ec-4602-bca1-c63049897773 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:01.798156 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea4c7f-90ec-4602-bca1-c63049897773 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:01.803660 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 940/1855] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:04:01 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:04:01.846578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:02.081085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:03.086869 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:03.098996 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2b03f-1231-453b-b9bb-d0250f7317c9 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:03.112076 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2b03f-1231-453b-b9bb-d0250f7317c9 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:03.117043 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06f18e-3608-4f53-bfba-7b5cd7e3b22d tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a12c984891b4d19bd61a77b2e4f593d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0d29c92b6644689b7ff6fc578c98576', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:03.117664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06f18e-3608-4f53-bfba-7b5cd7e3b22d tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:03.118012 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06f18e-3608-4f53-bfba-7b5cd7e3b22d tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:03.118465 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06f18e-3608-4f53-bfba-7b5cd7e3b22d tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:03.119865 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c06f18e-3608-4f53-bfba-7b5cd7e3b22d tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:04:03.120172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c06f18e-3608-4f53-bfba-7b5cd7e3b22d tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:03.121829 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c06f18e-3608-4f53-bfba-7b5cd7e3b22d tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:03.140664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 916/1856] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:04:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:04:03.144937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea4c7f-90ec-4602-bca1-c63049897773 service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:03.155237 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea4c7f-90ec-4602-bca1-c63049897773 service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:03.163737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b1803-aa71-4f63-a624-bca667942eeb tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a12c984891b4d19bd61a77b2e4f593d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0d29c92b6644689b7ff6fc578c98576', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:03.165150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b1803-aa71-4f63-a624-bca667942eeb tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:03.166469 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b1803-aa71-4f63-a624-bca667942eeb tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:03.166928 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b1803-aa71-4f63-a624-bca667942eeb tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:03.167828 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b1803-aa71-4f63-a624-bca667942eeb tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:03.170080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b1803-aa71-4f63-a624-bca667942eeb tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:03.352270 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 941/1857] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:04:03 2022] POST /identity/v3/projects => generated 355 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:04:03.357330 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06f18e-3608-4f53-bfba-7b5cd7e3b22d tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:03.367295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06f18e-3608-4f53-bfba-7b5cd7e3b22d tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:03.430745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18edf923-d231-4ec4-b427-1219735c922c tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a12c984891b4d19bd61a77b2e4f593d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0d29c92b6644689b7ff6fc578c98576', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:03.432143 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18edf923-d231-4ec4-b427-1219735c922c tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:03.432595 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18edf923-d231-4ec4-b427-1219735c922c tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:03.432698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18edf923-d231-4ec4-b427-1219735c922c tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:03.433326 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18edf923-d231-4ec4-b427-1219735c922c tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:04:03.433629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18edf923-d231-4ec4-b427-1219735c922c tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:03.435287 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18edf923-d231-4ec4-b427-1219735c922c tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:03.448731 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 917/1858] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:04:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:04:03.493120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b1803-aa71-4f63-a624-bca667942eeb tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:03.512922 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b1803-aa71-4f63-a624-bca667942eeb tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:03.519337 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ee6fd-3f9a-4bf2-b997-f4238e3a53c1 tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a12c984891b4d19bd61a77b2e4f593d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0d29c92b6644689b7ff6fc578c98576', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:03.520086 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ee6fd-3f9a-4bf2-b997-f4238e3a53c1 tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:03.520376 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ee6fd-3f9a-4bf2-b997-f4238e3a53c1 tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:03.520652 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ee6fd-3f9a-4bf2-b997-f4238e3a53c1 tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] PATH_INFO: `/v3/projects/5bc7523f1000420eb8e4099ba13dd7d3` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:03.521830 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ee6fd-3f9a-4bf2-b997-f4238e3a53c1 tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5bc7523f1000420eb8e4099ba13dd7d3)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:03.523765 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ee6fd-3f9a-4bf2-b997-f4238e3a53c1 tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:03.579639 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:03.664133 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a87ee6fd-3f9a-4bf2-b997-f4238e3a53c1 tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bc7523f1000420eb8e4099ba13dd7d3', 'request_id': 'req-a87ee6fd-3f9a-4bf2-b997-f4238e3a53c1'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:03.777777 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a87ee6fd-3f9a-4bf2-b997-f4238e3a53c1 tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bc7523f1000420eb8e4099ba13dd7d3', 'request_id': 'req-a87ee6fd-3f9a-4bf2-b997-f4238e3a53c1'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:03.783335 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a87ee6fd-3f9a-4bf2-b997-f4238e3a53c1 tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bc7523f1000420eb8e4099ba13dd7d3', 'request_id': 'req-a87ee6fd-3f9a-4bf2-b997-f4238e3a53c1'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:03.795067 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a87ee6fd-3f9a-4bf2-b997-f4238e3a53c1 tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] The token cache is being invalidate because project 5bc7523f1000420eb8e4099ba13dd7d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:04:03.795360 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a87ee6fd-3f9a-4bf2-b997-f4238e3a53c1 tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:03.806579 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a87ee6fd-3f9a-4bf2-b997-f4238e3a53c1 tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:03.806579 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 942/1859] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:04:03 2022] DELETE /identity/v3/projects/5bc7523f1000420eb8e4099ba13dd7d3 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:04.518602 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18edf923-d231-4ec4-b427-1219735c922c tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:04.520350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-18edf923-d231-4ec4-b427-1219735c922c tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:04.588526 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18edf923-d231-4ec4-b427-1219735c922c tempest-VolumeQuotaClassesTest-1029055990 tempest-VolumeQuotaClassesTest-1029055990-project-admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:04.645007 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f8e66-3911-4f43-874a-bd0baef6626b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:04.645668 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f8e66-3911-4f43-874a-bd0baef6626b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:04.645941 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f8e66-3911-4f43-874a-bd0baef6626b admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:04.646209 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f8e66-3911-4f43-874a-bd0baef6626b admin admin] PATH_INFO: `/v3/users/425bfdd105d0454d8e8111d88ed62f5e` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:04.647354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f8e66-3911-4f43-874a-bd0baef6626b admin admin] RBAC: Authorizing `identity:delete_user(user_id=425bfdd105d0454d8e8111d88ed62f5e)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:04.649238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f8e66-3911-4f43-874a-bd0baef6626b admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:04.654637 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:04.794815 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:04.877818 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5c9f8e66-3911-4f43-874a-bd0baef6626b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '425bfdd105d0454d8e8111d88ed62f5e'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:04.962201 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5c9f8e66-3911-4f43-874a-bd0baef6626b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '425bfdd105d0454d8e8111d88ed62f5e'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:04.963464 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5c9f8e66-3911-4f43-874a-bd0baef6626b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '425bfdd105d0454d8e8111d88ed62f5e'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:04.964175 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5c9f8e66-3911-4f43-874a-bd0baef6626b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '425bfdd105d0454d8e8111d88ed62f5e'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:04.977458 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5c9f8e66-3911-4f43-874a-bd0baef6626b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '425bfdd105d0454d8e8111d88ed62f5e'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:04.990037 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 918/1860] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:04:04 2022] DELETE /identity/v3/users/425bfdd105d0454d8e8111d88ed62f5e => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:04.995598 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07ef58-6a67-4190-8976-acc4378a5118 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:04.995909 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07ef58-6a67-4190-8976-acc4378a5118 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:04.996869 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07ef58-6a67-4190-8976-acc4378a5118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:05.018518 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-0e07ef58-6a67-4190-8976-acc4378a5118 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:05.019624 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-0e07ef58-6a67-4190-8976-acc4378a5118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:05.086039 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 943/1861] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:05.100225 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f8e66-3911-4f43-874a-bd0baef6626b admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:05.101768 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-5c9f8e66-3911-4f43-874a-bd0baef6626b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:05.164870 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f8e66-3911-4f43-874a-bd0baef6626b admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:05.168652 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620647e-352b-430b-95cd-783a81519681 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:05.169498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620647e-352b-430b-95cd-783a81519681 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:05.169788 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620647e-352b-430b-95cd-783a81519681 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:05.170075 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620647e-352b-430b-95cd-783a81519681 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:05.179838 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620647e-352b-430b-95cd-783a81519681 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:05.181831 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620647e-352b-430b-95cd-783a81519681 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:05.192441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 919/1862] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:04:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:04:05.511493 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:05.756503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:05.786913 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07ef58-6a67-4190-8976-acc4378a5118 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:05.788398 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-0e07ef58-6a67-4190-8976-acc4378a5118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:05.801151 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:05.803076 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07ef58-6a67-4190-8976-acc4378a5118 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:05.807878 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac6fee-59f7-44f4-8ffa-b1dd752b2dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:05.808532 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac6fee-59f7-44f4-8ffa-b1dd752b2dff admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:05.808813 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac6fee-59f7-44f4-8ffa-b1dd752b2dff admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:05.809263 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac6fee-59f7-44f4-8ffa-b1dd752b2dff admin admin] PATH_INFO: `/v3/projects/7d94de874a584fd19a55445674fdd8c5` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:05.810261 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac6fee-59f7-44f4-8ffa-b1dd752b2dff admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d94de874a584fd19a55445674fdd8c5)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:05.812221 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac6fee-59f7-44f4-8ffa-b1dd752b2dff admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:05.960245 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-4dac6fee-59f7-44f4-8ffa-b1dd752b2dff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d94de874a584fd19a55445674fdd8c5', 'request_id': 'req-4dac6fee-59f7-44f4-8ffa-b1dd752b2dff'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:06.053517 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-4dac6fee-59f7-44f4-8ffa-b1dd752b2dff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d94de874a584fd19a55445674fdd8c5', 'request_id': 'req-4dac6fee-59f7-44f4-8ffa-b1dd752b2dff'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:06.059951 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-4dac6fee-59f7-44f4-8ffa-b1dd752b2dff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d94de874a584fd19a55445674fdd8c5', 'request_id': 'req-4dac6fee-59f7-44f4-8ffa-b1dd752b2dff'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:06.074777 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-4dac6fee-59f7-44f4-8ffa-b1dd752b2dff admin admin] The token cache is being invalidate because project 7d94de874a584fd19a55445674fdd8c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:04:06.075298 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-4dac6fee-59f7-44f4-8ffa-b1dd752b2dff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:06.076201 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-4dac6fee-59f7-44f4-8ffa-b1dd752b2dff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:06.078585 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 944/1863] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:04:05 2022] DELETE /identity/v3/projects/7d94de874a584fd19a55445674fdd8c5 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:06.082507 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620647e-352b-430b-95cd-783a81519681 service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:06.094062 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620647e-352b-430b-95cd-783a81519681 service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:06.167300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f184a1-e833-4b07-9064-7491114e1342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:06.169587 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f184a1-e833-4b07-9064-7491114e1342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:06.169587 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f184a1-e833-4b07-9064-7491114e1342 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:06.169587 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f184a1-e833-4b07-9064-7491114e1342 admin admin] PATH_INFO: `/v3/users/1a12c984891b4d19bd61a77b2e4f593d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:06.169866 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f184a1-e833-4b07-9064-7491114e1342 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a12c984891b4d19bd61a77b2e4f593d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:06.171863 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f184a1-e833-4b07-9064-7491114e1342 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:06.411188 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-55f184a1-e833-4b07-9064-7491114e1342 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a12c984891b4d19bd61a77b2e4f593d'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:06.481103 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-55f184a1-e833-4b07-9064-7491114e1342 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a12c984891b4d19bd61a77b2e4f593d'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:06.483353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-55f184a1-e833-4b07-9064-7491114e1342 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a12c984891b4d19bd61a77b2e4f593d'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:06.483353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-55f184a1-e833-4b07-9064-7491114e1342 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a12c984891b4d19bd61a77b2e4f593d'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:06.499477 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-55f184a1-e833-4b07-9064-7491114e1342 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a12c984891b4d19bd61a77b2e4f593d'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:06.519021 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 920/1864] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:04:06 2022] DELETE /identity/v3/users/1a12c984891b4d19bd61a77b2e4f593d => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:06.746228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:07.477123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:07.576716 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:07.595551 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac6fee-59f7-44f4-8ffa-b1dd752b2dff admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:07.599218 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-4dac6fee-59f7-44f4-8ffa-b1dd752b2dff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:07.621374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac6fee-59f7-44f4-8ffa-b1dd752b2dff admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:07.767405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8231d2-6545-4635-a1de-20b394df2282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:07.768056 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8231d2-6545-4635-a1de-20b394df2282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:07.768335 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8231d2-6545-4635-a1de-20b394df2282 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:07.768609 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8231d2-6545-4635-a1de-20b394df2282 admin admin] PATH_INFO: `/v3/projects/c0d29c92b6644689b7ff6fc578c98576` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:07.769943 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8231d2-6545-4635-a1de-20b394df2282 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0d29c92b6644689b7ff6fc578c98576)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:07.774125 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8231d2-6545-4635-a1de-20b394df2282 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:07.918823 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-5c8231d2-6545-4635-a1de-20b394df2282 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0d29c92b6644689b7ff6fc578c98576', 'request_id': 'req-5c8231d2-6545-4635-a1de-20b394df2282'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:07.991009 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-5c8231d2-6545-4635-a1de-20b394df2282 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0d29c92b6644689b7ff6fc578c98576', 'request_id': 'req-5c8231d2-6545-4635-a1de-20b394df2282'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:08.001908 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-5c8231d2-6545-4635-a1de-20b394df2282 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0d29c92b6644689b7ff6fc578c98576', 'request_id': 'req-5c8231d2-6545-4635-a1de-20b394df2282'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:08.022376 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-5c8231d2-6545-4635-a1de-20b394df2282 admin admin] The token cache is being invalidate because project c0d29c92b6644689b7ff6fc578c98576 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:04:08.022760 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-5c8231d2-6545-4635-a1de-20b394df2282 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:08.023443 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-5c8231d2-6545-4635-a1de-20b394df2282 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:08.030150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 945/1865] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:04:07 2022] DELETE /identity/v3/projects/c0d29c92b6644689b7ff6fc578c98576 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:08.038655 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbe037-f693-4c75-a4a4-96323753dbe2 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:08.038655 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbe037-f693-4c75-a4a4-96323753dbe2 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:08.038655 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbe037-f693-4c75-a4a4-96323753dbe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:08.104091 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-4cdbe037-f693-4c75-a4a4-96323753dbe2 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:08.104091 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-4cdbe037-f693-4c75-a4a4-96323753dbe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:08.169612 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 921/1866] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:08.177858 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04811cf-bbe1-4cb6-b86c-319ae8c68095 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:08.178169 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04811cf-bbe1-4cb6-b86c-319ae8c68095 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:08.178489 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04811cf-bbe1-4cb6-b86c-319ae8c68095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:08.206275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-a04811cf-bbe1-4cb6-b86c-319ae8c68095 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:08.207640 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a04811cf-bbe1-4cb6-b86c-319ae8c68095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:08.220621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 946/1867] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:04:08.222282 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdbe037-f693-4c75-a4a4-96323753dbe2 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:08.237681 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdbe037-f693-4c75-a4a4-96323753dbe2 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:08.243110 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f647d97-7033-411a-909b-45fca4dea3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:08.244505 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f647d97-7033-411a-909b-45fca4dea3e3 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:08.244505 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f647d97-7033-411a-909b-45fca4dea3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:08.244505 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f647d97-7033-411a-909b-45fca4dea3e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:08.245410 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f647d97-7033-411a-909b-45fca4dea3e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:04:08.245715 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f647d97-7033-411a-909b-45fca4dea3e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:08.247628 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f647d97-7033-411a-909b-45fca4dea3e3 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:08.271234 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 922/1868] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:04:08.276017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04811cf-bbe1-4cb6-b86c-319ae8c68095 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:08.287996 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04811cf-bbe1-4cb6-b86c-319ae8c68095 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:08.293723 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1cca8-21b2-4ae5-90ba-1433405330db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:08.294845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1cca8-21b2-4ae5-90ba-1433405330db admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:08.294845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1cca8-21b2-4ae5-90ba-1433405330db admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:08.295109 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1cca8-21b2-4ae5-90ba-1433405330db admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:08.297723 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1cca8-21b2-4ae5-90ba-1433405330db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:08.298033 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1cca8-21b2-4ae5-90ba-1433405330db admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:08.435561 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1aeea-5983-4670-aae7-6895f3423f23 None None] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:08.435924 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1aeea-5983-4670-aae7-6895f3423f23 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:08.436270 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1aeea-5983-4670-aae7-6895f3423f23 None None] PATH_INFO: `/` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:08.437693 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 923/1869] 10.222.0.16 () {54 vars in 921 bytes} [Wed Aug 31 11:04:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:04:08.444974 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340728c5-e5c8-43f2-915c-a445124f7b8e None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:08.447592 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340728c5-e5c8-43f2-915c-a445124f7b8e None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:08.447592 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340728c5-e5c8-43f2-915c-a445124f7b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:08.481976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 947/1870] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:04:08 2022] POST /identity/v3/projects => generated 367 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:04:08.486115 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-340728c5-e5c8-43f2-915c-a445124f7b8e None None] MFA Rules not processed for user `f11971a594014d86968c3356d11d42f7`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:08.493658 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1cca8-21b2-4ae5-90ba-1433405330db admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:08.493818 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-340728c5-e5c8-43f2-915c-a445124f7b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:08.560709 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1cca8-21b2-4ae5-90ba-1433405330db admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:08.566215 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68a6a4-f645-4519-960d-d5f4440d8243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:08.567104 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68a6a4-f645-4519-960d-d5f4440d8243 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:08.567388 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68a6a4-f645-4519-960d-d5f4440d8243 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:08.567664 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68a6a4-f645-4519-960d-d5f4440d8243 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:08.568337 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68a6a4-f645-4519-960d-d5f4440d8243 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:08.568650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 924/1871] 10.222.0.16 () {58 vars in 1018 bytes} [Wed Aug 31 11:04:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:04:08.570491 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68a6a4-f645-4519-960d-d5f4440d8243 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:08.587653 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340728c5-e5c8-43f2-915c-a445124f7b8e None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:08.591783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-340728c5-e5c8-43f2-915c-a445124f7b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:08.612964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340728c5-e5c8-43f2-915c-a445124f7b8e None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:08.620062 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a49aa7-2de0-41b2-9ebd-ee69dbd1add5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:08.621182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a49aa7-2de0-41b2-9ebd-ee69dbd1add5 service nova] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:08.621483 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a49aa7-2de0-41b2-9ebd-ee69dbd1add5 service nova] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:08.621751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a49aa7-2de0-41b2-9ebd-ee69dbd1add5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:08.632395 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a49aa7-2de0-41b2-9ebd-ee69dbd1add5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:08.634423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a49aa7-2de0-41b2-9ebd-ee69dbd1add5 service nova] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:08.646949 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 925/1872] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:04:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:04:08.660907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 948/1873] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:04:08 2022] POST /identity/v3/users => generated 284 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:04:08.668738 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a49aa7-2de0-41b2-9ebd-ee69dbd1add5 service nova] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:08.697664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a49aa7-2de0-41b2-9ebd-ee69dbd1add5 service nova] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:08.707139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa376194-ecae-4323-8d12-cf2f1aefcee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:08.709363 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa376194-ecae-4323-8d12-cf2f1aefcee3 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:08.709692 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa376194-ecae-4323-8d12-cf2f1aefcee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:08.710109 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa376194-ecae-4323-8d12-cf2f1aefcee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:08.710810 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa376194-ecae-4323-8d12-cf2f1aefcee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:08.712599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa376194-ecae-4323-8d12-cf2f1aefcee3 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:08.731066 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:08.745712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 926/1874] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:04:08.753030 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68a6a4-f645-4519-960d-d5f4440d8243 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:08.766562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68a6a4-f645-4519-960d-d5f4440d8243 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:08.770757 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc9811-8aa7-4b37-866e-f94bd7695fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:08.771386 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc9811-8aa7-4b37-866e-f94bd7695fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:08.771643 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc9811-8aa7-4b37-866e-f94bd7695fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:08.773298 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc9811-8aa7-4b37-866e-f94bd7695fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:08.774496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc9811-8aa7-4b37-866e-f94bd7695fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:08.777184 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc9811-8aa7-4b37-866e-f94bd7695fb8 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:08.789029 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 949/1875] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:04:08.794666 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa376194-ecae-4323-8d12-cf2f1aefcee3 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:08.804965 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa376194-ecae-4323-8d12-cf2f1aefcee3 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:08.810009 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec828962-5386-4bc7-82b8-4f7e2303e9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:08.810706 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec828962-5386-4bc7-82b8-4f7e2303e9cf admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:08.810975 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec828962-5386-4bc7-82b8-4f7e2303e9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:08.811242 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec828962-5386-4bc7-82b8-4f7e2303e9cf admin admin] PATH_INFO: `/v3/projects/65c9421921124f1888355208c07802c3/users/03ddac5a102b4286a993d17d1fe939a1/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:08.828462 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec828962-5386-4bc7-82b8-4f7e2303e9cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=65c9421921124f1888355208c07802c3, user_id=03ddac5a102b4286a993d17d1fe939a1, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:08.831145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec828962-5386-4bc7-82b8-4f7e2303e9cf admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:08.921856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 927/1876] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:04:08 2022] PUT /identity/v3/projects/65c9421921124f1888355208c07802c3/users/03ddac5a102b4286a993d17d1fe939a1/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:08.936820 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b42d9-4869-45bc-ae4a-c66abaadeb2a None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:08.937176 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b42d9-4869-45bc-ae4a-c66abaadeb2a None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:08.937520 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b42d9-4869-45bc-ae4a-c66abaadeb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:08.969490 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-335b42d9-4869-45bc-ae4a-c66abaadeb2a None None] MFA Rules not processed for user `03ddac5a102b4286a993d17d1fe939a1`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:08.972773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-335b42d9-4869-45bc-ae4a-c66abaadeb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:09.046672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 950/1877] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:09.051369 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec828962-5386-4bc7-82b8-4f7e2303e9cf admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:09.116802 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:09.123187 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec828962-5386-4bc7-82b8-4f7e2303e9cf admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:09.128149 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c198f-39b2-49f8-a12d-f00dc727c5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:09.128764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c198f-39b2-49f8-a12d-f00dc727c5c6 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:09.129043 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c198f-39b2-49f8-a12d-f00dc727c5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:09.129313 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c198f-39b2-49f8-a12d-f00dc727c5c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:09.129945 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c198f-39b2-49f8-a12d-f00dc727c5c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:09.132049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c198f-39b2-49f8-a12d-f00dc727c5c6 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:09.243790 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 928/1878] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:04:09 2022] POST /identity/v3/projects => generated 367 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:04:09.249148 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b42d9-4869-45bc-ae4a-c66abaadeb2a None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:09.259739 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b42d9-4869-45bc-ae4a-c66abaadeb2a None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:09.299710 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:09.317447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07b13e-cc71-4e1b-9bd6-9017cf607d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:09.318058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07b13e-cc71-4e1b-9bd6-9017cf607d51 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:09.318338 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07b13e-cc71-4e1b-9bd6-9017cf607d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:09.318759 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07b13e-cc71-4e1b-9bd6-9017cf607d51 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:09.319490 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07b13e-cc71-4e1b-9bd6-9017cf607d51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:09.321452 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07b13e-cc71-4e1b-9bd6-9017cf607d51 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:09.439996 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 951/1879] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:04:09 2022] POST /identity/v3/users => generated 290 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:04:09.444330 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c198f-39b2-49f8-a12d-f00dc727c5c6 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:09.457873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c198f-39b2-49f8-a12d-f00dc727c5c6 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:09.465093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93aafc-dd34-4141-a54b-a639f3ca9720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:09.465740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93aafc-dd34-4141-a54b-a639f3ca9720 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:09.466018 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93aafc-dd34-4141-a54b-a639f3ca9720 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:09.466286 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93aafc-dd34-4141-a54b-a639f3ca9720 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:09.467071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b93aafc-dd34-4141-a54b-a639f3ca9720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:09.468649 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b93aafc-dd34-4141-a54b-a639f3ca9720 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:09.481327 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 929/1880] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:04:09.485573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07b13e-cc71-4e1b-9bd6-9017cf607d51 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:09.497974 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07b13e-cc71-4e1b-9bd6-9017cf607d51 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:09.501213 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5908aab8-0e2d-40c2-bee6-d6151aa2252f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:09.501708 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908aab8-0e2d-40c2-bee6-d6151aa2252f admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:09.501991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908aab8-0e2d-40c2-bee6-d6151aa2252f admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:09.502276 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908aab8-0e2d-40c2-bee6-d6151aa2252f admin admin] PATH_INFO: `/v3/projects/c332614d068e47b09a87ebb4907fe7e4/users/945d1f50b52c4a8f9084202fda09607e/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:09.521802 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5908aab8-0e2d-40c2-bee6-d6151aa2252f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c332614d068e47b09a87ebb4907fe7e4, user_id=945d1f50b52c4a8f9084202fda09607e, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:09.525244 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5908aab8-0e2d-40c2-bee6-d6151aa2252f admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:09.601128 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 952/1881] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:04:09 2022] PUT /identity/v3/projects/c332614d068e47b09a87ebb4907fe7e4/users/945d1f50b52c4a8f9084202fda09607e/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:09.607904 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93aafc-dd34-4141-a54b-a639f3ca9720 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:09.617860 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93aafc-dd34-4141-a54b-a639f3ca9720 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:09.668909 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:09.675534 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c704f33-eb61-4fa3-8631-205fc8a2aaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:09.676151 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c704f33-eb61-4fa3-8631-205fc8a2aaa5 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:09.676428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c704f33-eb61-4fa3-8631-205fc8a2aaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:09.676699 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c704f33-eb61-4fa3-8631-205fc8a2aaa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:09.677339 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c704f33-eb61-4fa3-8631-205fc8a2aaa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:09.678962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c704f33-eb61-4fa3-8631-205fc8a2aaa5 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:09.690371 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 930/1882] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:04:09.694756 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5908aab8-0e2d-40c2-bee6-d6151aa2252f admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:09.706442 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5908aab8-0e2d-40c2-bee6-d6151aa2252f admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:09.711144 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c7b2a-ff4a-441a-af71-50416a8a0057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:09.711814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c7b2a-ff4a-441a-af71-50416a8a0057 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:09.712092 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c7b2a-ff4a-441a-af71-50416a8a0057 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:09.712360 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c7b2a-ff4a-441a-af71-50416a8a0057 admin admin] PATH_INFO: `/v3/projects/c332614d068e47b09a87ebb4907fe7e4/users/945d1f50b52c4a8f9084202fda09607e/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:09.715128 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c7b2a-ff4a-441a-af71-50416a8a0057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c332614d068e47b09a87ebb4907fe7e4, user_id=945d1f50b52c4a8f9084202fda09607e, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:09.717793 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c7b2a-ff4a-441a-af71-50416a8a0057 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:09.730240 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 953/1883] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:04:09 2022] PUT /identity/v3/projects/c332614d068e47b09a87ebb4907fe7e4/users/945d1f50b52c4a8f9084202fda09607e/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:09.740449 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997ca24-30fa-4a95-bb96-1505661b1ac7 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:09.740739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997ca24-30fa-4a95-bb96-1505661b1ac7 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:09.741014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997ca24-30fa-4a95-bb96-1505661b1ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:09.782186 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-e997ca24-30fa-4a95-bb96-1505661b1ac7 None None] MFA Rules not processed for user `945d1f50b52c4a8f9084202fda09607e`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:09.784011 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-e997ca24-30fa-4a95-bb96-1505661b1ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:09.879500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 931/1884] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:09 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:09.892033 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c7b2a-ff4a-441a-af71-50416a8a0057 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:09.893785 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-965c7b2a-ff4a-441a-af71-50416a8a0057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:09.958407 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c7b2a-ff4a-441a-af71-50416a8a0057 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:09.962475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d50f9d-a387-4218-a67d-fae6482c07c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:09.963135 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d50f9d-a387-4218-a67d-fae6482c07c1 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:09.963416 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d50f9d-a387-4218-a67d-fae6482c07c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:09.963909 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d50f9d-a387-4218-a67d-fae6482c07c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:09.974779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d50f9d-a387-4218-a67d-fae6482c07c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:09.976390 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d50f9d-a387-4218-a67d-fae6482c07c1 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:09.987901 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 954/1885] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:04:09 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:04:10.217960 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:10.221179 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e997ca24-30fa-4a95-bb96-1505661b1ac7 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:10.232157 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e997ca24-30fa-4a95-bb96-1505661b1ac7 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:10.235701 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c2208b-156e-48ef-bdac-644749e0b64e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:10.236309 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2208b-156e-48ef-bdac-644749e0b64e service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:10.236587 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2208b-156e-48ef-bdac-644749e0b64e service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:10.236859 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2208b-156e-48ef-bdac-644749e0b64e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:10.295120 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c2208b-156e-48ef-bdac-644749e0b64e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:10.296557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c2208b-156e-48ef-bdac-644749e0b64e service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:10.305732 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 932/1886] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:04:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:04:10.408635 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:11.683748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:11.897761 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:12.185637 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:12.664348 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:12.900766 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:12.958044 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:13.122872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:13.184423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:13.671621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:13.860188 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:13.887476 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:14.140579 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:14.169376 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=87,keystone:INSERT=7,keystone:DELETE=6 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:14.611427 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:14.732238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d50f9d-a387-4218-a67d-fae6482c07c1 service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:14.740493 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d50f9d-a387-4218-a67d-fae6482c07c1 service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:14.744162 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1084aa3-4cb4-4cb3-88f9-8868f2da0b2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:14.745393 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1084aa3-4cb4-4cb3-88f9-8868f2da0b2f service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:14.746164 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1084aa3-4cb4-4cb3-88f9-8868f2da0b2f service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:14.746164 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1084aa3-4cb4-4cb3-88f9-8868f2da0b2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:14.747856 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-b1084aa3-4cb4-4cb3-88f9-8868f2da0b2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:14.809215 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1084aa3-4cb4-4cb3-88f9-8868f2da0b2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:14.810703 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1084aa3-4cb4-4cb3-88f9-8868f2da0b2f service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:14.819674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 955/1887] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:04:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:04:15.006571 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:15.794648 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:16.040455 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:16.425239 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:16.648140 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:17.532360 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c2208b-156e-48ef-bdac-644749e0b64e service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:17.533776 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-87c2208b-156e-48ef-bdac-644749e0b64e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:17.546370 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c2208b-156e-48ef-bdac-644749e0b64e service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:17.550182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d85f6-f658-4767-a842-5dfd2c8dfeb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:17.550843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d85f6-f658-4767-a842-5dfd2c8dfeb6 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:17.551117 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d85f6-f658-4767-a842-5dfd2c8dfeb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:17.551383 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d85f6-f658-4767-a842-5dfd2c8dfeb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:17.553231 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-eb1d85f6-f658-4767-a842-5dfd2c8dfeb6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:17.619354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d85f6-f658-4767-a842-5dfd2c8dfeb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:17.621130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d85f6-f658-4767-a842-5dfd2c8dfeb6 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:17.633030 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 933/1888] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:04:17 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:04:21.705703 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1084aa3-4cb4-4cb3-88f9-8868f2da0b2f service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:21.833613 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1084aa3-4cb4-4cb3-88f9-8868f2da0b2f service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:21.839588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb74c8-004c-4227-8a5c-007359969a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:21.840277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb74c8-004c-4227-8a5c-007359969a46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:21.840743 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb74c8-004c-4227-8a5c-007359969a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:21.841015 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb74c8-004c-4227-8a5c-007359969a46 admin admin] PATH_INFO: `/v3/users/03ddac5a102b4286a993d17d1fe939a1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:21.842301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb74c8-004c-4227-8a5c-007359969a46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03ddac5a102b4286a993d17d1fe939a1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:21.844262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb74c8-004c-4227-8a5c-007359969a46 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:22.138875 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a7eb74c8-004c-4227-8a5c-007359969a46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03ddac5a102b4286a993d17d1fe939a1'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:22.158367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a7eb74c8-004c-4227-8a5c-007359969a46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03ddac5a102b4286a993d17d1fe939a1'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:22.165881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a7eb74c8-004c-4227-8a5c-007359969a46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03ddac5a102b4286a993d17d1fe939a1'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:22.236393 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a7eb74c8-004c-4227-8a5c-007359969a46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03ddac5a102b4286a993d17d1fe939a1'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:22.237016 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a7eb74c8-004c-4227-8a5c-007359969a46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03ddac5a102b4286a993d17d1fe939a1'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:22.243533 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 956/1889] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:04:21 2022] DELETE /identity/v3/users/03ddac5a102b4286a993d17d1fe939a1 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:22.249233 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5f8a1-2f27-4664-a8f2-6174e7afacf1 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:22.249788 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5f8a1-2f27-4664-a8f2-6174e7afacf1 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:22.249788 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5f8a1-2f27-4664-a8f2-6174e7afacf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:22.269808 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-10b5f8a1-2f27-4664-a8f2-6174e7afacf1 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:22.271191 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-10b5f8a1-2f27-4664-a8f2-6174e7afacf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:22.332434 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 934/1890] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:22.342915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb74c8-004c-4227-8a5c-007359969a46 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:22.344568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a7eb74c8-004c-4227-8a5c-007359969a46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:22.428511 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb74c8-004c-4227-8a5c-007359969a46 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:22.432524 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de487caa-53d9-49f9-9d82-7f316b3d354b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:22.433128 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de487caa-53d9-49f9-9d82-7f316b3d354b service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:22.433425 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de487caa-53d9-49f9-9d82-7f316b3d354b service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:22.433700 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de487caa-53d9-49f9-9d82-7f316b3d354b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:22.442820 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de487caa-53d9-49f9-9d82-7f316b3d354b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:22.444276 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de487caa-53d9-49f9-9d82-7f316b3d354b service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:22.448144 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:22.454103 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 957/1891] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:04:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:04:22.975508 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5f8a1-2f27-4664-a8f2-6174e7afacf1 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:22.978699 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-10b5f8a1-2f27-4664-a8f2-6174e7afacf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:22.995872 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5f8a1-2f27-4664-a8f2-6174e7afacf1 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:23.002273 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b00b8-b7ae-4795-b477-acf7542bdcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:23.003064 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b00b8-b7ae-4795-b477-acf7542bdcfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:23.003410 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b00b8-b7ae-4795-b477-acf7542bdcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:23.003748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b00b8-b7ae-4795-b477-acf7542bdcfa admin admin] PATH_INFO: `/v3/projects/65c9421921124f1888355208c07802c3` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:23.005272 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b00b8-b7ae-4795-b477-acf7542bdcfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=65c9421921124f1888355208c07802c3)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:23.008321 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b00b8-b7ae-4795-b477-acf7542bdcfa admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:23.124696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-cf0b00b8-b7ae-4795-b477-acf7542bdcfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65c9421921124f1888355208c07802c3', 'request_id': 'req-cf0b00b8-b7ae-4795-b477-acf7542bdcfa'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:23.130815 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-cf0b00b8-b7ae-4795-b477-acf7542bdcfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65c9421921124f1888355208c07802c3', 'request_id': 'req-cf0b00b8-b7ae-4795-b477-acf7542bdcfa'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:23.132818 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:23.137326 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-cf0b00b8-b7ae-4795-b477-acf7542bdcfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65c9421921124f1888355208c07802c3', 'request_id': 'req-cf0b00b8-b7ae-4795-b477-acf7542bdcfa'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:23.265866 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-cf0b00b8-b7ae-4795-b477-acf7542bdcfa admin admin] The token cache is being invalidate because project 65c9421921124f1888355208c07802c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:04:23.266520 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-cf0b00b8-b7ae-4795-b477-acf7542bdcfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:23.267030 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-cf0b00b8-b7ae-4795-b477-acf7542bdcfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:23.269434 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 935/1892] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:04:22 2022] DELETE /identity/v3/projects/65c9421921124f1888355208c07802c3 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:23.279753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de487caa-53d9-49f9-9d82-7f316b3d354b service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:23.289675 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de487caa-53d9-49f9-9d82-7f316b3d354b service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:23.348371 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a94307-4aa6-41ea-bb8b-acce78061e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:23.349003 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a94307-4aa6-41ea-bb8b-acce78061e26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:23.349282 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a94307-4aa6-41ea-bb8b-acce78061e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:23.349554 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a94307-4aa6-41ea-bb8b-acce78061e26 admin admin] PATH_INFO: `/v3/users/945d1f50b52c4a8f9084202fda09607e` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:23.350704 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a94307-4aa6-41ea-bb8b-acce78061e26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=945d1f50b52c4a8f9084202fda09607e)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:23.352634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a94307-4aa6-41ea-bb8b-acce78061e26 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:23.790020 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-62a94307-4aa6-41ea-bb8b-acce78061e26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '945d1f50b52c4a8f9084202fda09607e'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:23.810702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-62a94307-4aa6-41ea-bb8b-acce78061e26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '945d1f50b52c4a8f9084202fda09607e'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:23.825083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-62a94307-4aa6-41ea-bb8b-acce78061e26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '945d1f50b52c4a8f9084202fda09607e'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:23.961104 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-62a94307-4aa6-41ea-bb8b-acce78061e26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '945d1f50b52c4a8f9084202fda09607e'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:23.961871 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-62a94307-4aa6-41ea-bb8b-acce78061e26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '945d1f50b52c4a8f9084202fda09607e'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:23.970050 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 958/1893] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:04:23 2022] DELETE /identity/v3/users/945d1f50b52c4a8f9084202fda09607e => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:24.513590 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b00b8-b7ae-4795-b477-acf7542bdcfa admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:24.515164 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-cf0b00b8-b7ae-4795-b477-acf7542bdcfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:24.531972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b00b8-b7ae-4795-b477-acf7542bdcfa admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:24.595924 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab122611-aa1c-4305-a9a7-a0dd09923258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:24.596591 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab122611-aa1c-4305-a9a7-a0dd09923258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:24.596874 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab122611-aa1c-4305-a9a7-a0dd09923258 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:24.597197 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab122611-aa1c-4305-a9a7-a0dd09923258 admin admin] PATH_INFO: `/v3/projects/c332614d068e47b09a87ebb4907fe7e4` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:24.603231 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab122611-aa1c-4305-a9a7-a0dd09923258 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c332614d068e47b09a87ebb4907fe7e4)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:24.603231 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab122611-aa1c-4305-a9a7-a0dd09923258 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:24.721963 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-ab122611-aa1c-4305-a9a7-a0dd09923258 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c332614d068e47b09a87ebb4907fe7e4', 'request_id': 'req-ab122611-aa1c-4305-a9a7-a0dd09923258'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:24.728349 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-ab122611-aa1c-4305-a9a7-a0dd09923258 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c332614d068e47b09a87ebb4907fe7e4', 'request_id': 'req-ab122611-aa1c-4305-a9a7-a0dd09923258'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:24.733914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-ab122611-aa1c-4305-a9a7-a0dd09923258 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c332614d068e47b09a87ebb4907fe7e4', 'request_id': 'req-ab122611-aa1c-4305-a9a7-a0dd09923258'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:24.823823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-ab122611-aa1c-4305-a9a7-a0dd09923258 admin admin] The token cache is being invalidate because project c332614d068e47b09a87ebb4907fe7e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:04:24.824123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-ab122611-aa1c-4305-a9a7-a0dd09923258 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:24.824751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-ab122611-aa1c-4305-a9a7-a0dd09923258 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:24.827204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 936/1894] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:04:24 2022] DELETE /identity/v3/projects/c332614d068e47b09a87ebb4907fe7e4 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:24.833484 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351005a1-fe85-4eff-85ca-6ad5c3356031 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:24.834293 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351005a1-fe85-4eff-85ca-6ad5c3356031 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:24.834675 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351005a1-fe85-4eff-85ca-6ad5c3356031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:24.863489 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-351005a1-fe85-4eff-85ca-6ad5c3356031 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:24.864718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-351005a1-fe85-4eff-85ca-6ad5c3356031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:24.936165 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 959/1895] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:24.946791 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73847142-e699-413b-922d-5fccb33ea6c5 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:24.947324 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73847142-e699-413b-922d-5fccb33ea6c5 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:24.947816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73847142-e699-413b-922d-5fccb33ea6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:24.973980 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-73847142-e699-413b-922d-5fccb33ea6c5 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:24.975283 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-73847142-e699-413b-922d-5fccb33ea6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:24.984628 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 937/1896] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:24.989365 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351005a1-fe85-4eff-85ca-6ad5c3356031 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:25.000329 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351005a1-fe85-4eff-85ca-6ad5c3356031 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:25.005105 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7c235-9fc2-4610-a51c-f03417fae17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:25.005709 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7c235-9fc2-4610-a51c-f03417fae17b admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:25.005986 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7c235-9fc2-4610-a51c-f03417fae17b admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:25.006255 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7c235-9fc2-4610-a51c-f03417fae17b admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:25.006933 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7c235-9fc2-4610-a51c-f03417fae17b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:04:25.007337 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7c235-9fc2-4610-a51c-f03417fae17b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:25.009002 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7c235-9fc2-4610-a51c-f03417fae17b admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:25.021048 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 960/1897] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:04:25.026310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73847142-e699-413b-922d-5fccb33ea6c5 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:25.040683 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73847142-e699-413b-922d-5fccb33ea6c5 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:25.048070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece683e-5f9f-4355-bd47-60260435939a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:25.048999 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece683e-5f9f-4355-bd47-60260435939a admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:25.049440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece683e-5f9f-4355-bd47-60260435939a admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:25.049872 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece683e-5f9f-4355-bd47-60260435939a admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:25.051061 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ece683e-5f9f-4355-bd47-60260435939a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:25.054752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ece683e-5f9f-4355-bd47-60260435939a admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:25.186894 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 938/1898] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:04:25 2022] POST /identity/v3/projects => generated 371 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:04:25.191162 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7c235-9fc2-4610-a51c-f03417fae17b admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:25.198187 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:25.202254 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7c235-9fc2-4610-a51c-f03417fae17b admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:25.275285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca3665-407d-44e3-ad7f-fd01d4d872da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:25.275285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca3665-407d-44e3-ad7f-fd01d4d872da admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:25.275285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca3665-407d-44e3-ad7f-fd01d4d872da admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:25.275285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca3665-407d-44e3-ad7f-fd01d4d872da admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:25.277300 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca3665-407d-44e3-ad7f-fd01d4d872da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:25.277300 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca3665-407d-44e3-ad7f-fd01d4d872da admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:25.402462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 961/1899] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:04:25 2022] POST /identity/v3/users => generated 286 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:04:25.407566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece683e-5f9f-4355-bd47-60260435939a admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:25.425285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece683e-5f9f-4355-bd47-60260435939a admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:25.430498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ecdc3-ed51-4d8e-8049-f2cad001613c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:25.431120 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ecdc3-ed51-4d8e-8049-f2cad001613c admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:25.431395 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ecdc3-ed51-4d8e-8049-f2cad001613c admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:25.431666 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ecdc3-ed51-4d8e-8049-f2cad001613c admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:25.433071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ecdc3-ed51-4d8e-8049-f2cad001613c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:25.434837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ecdc3-ed51-4d8e-8049-f2cad001613c admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:25.445374 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 939/1900] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:04:25.449838 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca3665-407d-44e3-ad7f-fd01d4d872da admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:25.460376 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca3665-407d-44e3-ad7f-fd01d4d872da admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:25.465744 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982e46d-9c94-4d11-89fc-344619996dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:25.466340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982e46d-9c94-4d11-89fc-344619996dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:25.466648 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982e46d-9c94-4d11-89fc-344619996dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:25.466917 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982e46d-9c94-4d11-89fc-344619996dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:25.467576 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982e46d-9c94-4d11-89fc-344619996dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:25.469146 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982e46d-9c94-4d11-89fc-344619996dc1 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:25.480358 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 962/1901] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:04:25.484554 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ecdc3-ed51-4d8e-8049-f2cad001613c admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:25.494854 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ecdc3-ed51-4d8e-8049-f2cad001613c admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:25.499512 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26b739-3980-449b-9acc-2f746aebeda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:25.501743 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26b739-3980-449b-9acc-2f746aebeda6 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:25.501743 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26b739-3980-449b-9acc-2f746aebeda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:25.501743 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26b739-3980-449b-9acc-2f746aebeda6 admin admin] PATH_INFO: `/v3/projects/25b1d4822e9b421bad55479bd780ccee/users/abc991b61c6144ee92c993ed61adc4fc/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:25.520412 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f26b739-3980-449b-9acc-2f746aebeda6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25b1d4822e9b421bad55479bd780ccee, user_id=abc991b61c6144ee92c993ed61adc4fc, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:25.523116 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f26b739-3980-449b-9acc-2f746aebeda6 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:25.624922 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 940/1902] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:04:25 2022] PUT /identity/v3/projects/25b1d4822e9b421bad55479bd780ccee/users/abc991b61c6144ee92c993ed61adc4fc/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:25.637721 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2546af-e807-4e18-8e1c-3e5aff3e30b1 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:25.638014 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2546af-e807-4e18-8e1c-3e5aff3e30b1 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:25.638282 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2546af-e807-4e18-8e1c-3e5aff3e30b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:25.670600 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-1c2546af-e807-4e18-8e1c-3e5aff3e30b1 None None] MFA Rules not processed for user `abc991b61c6144ee92c993ed61adc4fc`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:25.671827 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-1c2546af-e807-4e18-8e1c-3e5aff3e30b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:25.741078 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 963/1903] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:04:25.752462 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26b739-3980-449b-9acc-2f746aebeda6 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:25.757168 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-7f26b739-3980-449b-9acc-2f746aebeda6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:25.769203 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:25.814632 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26b739-3980-449b-9acc-2f746aebeda6 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:25.818199 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970db7dd-54d5-4852-bf1c-8a4e3d951907 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:25.818830 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970db7dd-54d5-4852-bf1c-8a4e3d951907 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:25.819106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970db7dd-54d5-4852-bf1c-8a4e3d951907 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:25.819373 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970db7dd-54d5-4852-bf1c-8a4e3d951907 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:25.828407 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970db7dd-54d5-4852-bf1c-8a4e3d951907 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:25.830090 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970db7dd-54d5-4852-bf1c-8a4e3d951907 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:25.840439 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 941/1904] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:04:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:04:25.865133 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2546af-e807-4e18-8e1c-3e5aff3e30b1 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:25.934133 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2546af-e807-4e18-8e1c-3e5aff3e30b1 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:25.938853 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a073c716-3f2b-40e3-90d0-5c2d64f1d6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:25.939483 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073c716-3f2b-40e3-90d0-5c2d64f1d6ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:25.939762 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073c716-3f2b-40e3-90d0-5c2d64f1d6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:25.940038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073c716-3f2b-40e3-90d0-5c2d64f1d6ae admin admin] PATH_INFO: `/v3/users/abc991b61c6144ee92c993ed61adc4fc` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:25.941201 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a073c716-3f2b-40e3-90d0-5c2d64f1d6ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=abc991b61c6144ee92c993ed61adc4fc)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:25.943156 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a073c716-3f2b-40e3-90d0-5c2d64f1d6ae admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:25.957050 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:26.231794 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a073c716-3f2b-40e3-90d0-5c2d64f1d6ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abc991b61c6144ee92c993ed61adc4fc'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:26.243294 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a073c716-3f2b-40e3-90d0-5c2d64f1d6ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abc991b61c6144ee92c993ed61adc4fc'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:26.252486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a073c716-3f2b-40e3-90d0-5c2d64f1d6ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abc991b61c6144ee92c993ed61adc4fc'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:26.391047 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a073c716-3f2b-40e3-90d0-5c2d64f1d6ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abc991b61c6144ee92c993ed61adc4fc'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:26.391660 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a073c716-3f2b-40e3-90d0-5c2d64f1d6ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abc991b61c6144ee92c993ed61adc4fc'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:26.398477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 964/1905] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:04:25 2022] DELETE /identity/v3/users/abc991b61c6144ee92c993ed61adc4fc => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:26.403749 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e23db-add9-46c2-aaae-8d970bb5e224 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:26.404215 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e23db-add9-46c2-aaae-8d970bb5e224 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:26.404530 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e23db-add9-46c2-aaae-8d970bb5e224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:26.433587 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-670e23db-add9-46c2-aaae-8d970bb5e224 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:26.434888 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-670e23db-add9-46c2-aaae-8d970bb5e224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:26.535753 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 942/1906] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:04:26.541930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a073c716-3f2b-40e3-90d0-5c2d64f1d6ae admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:26.543637 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a073c716-3f2b-40e3-90d0-5c2d64f1d6ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:26.609784 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a073c716-3f2b-40e3-90d0-5c2d64f1d6ae admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:26.613545 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f791d93-bbaa-48ff-96d4-16cd24aba377 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:26.614426 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f791d93-bbaa-48ff-96d4-16cd24aba377 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:26.614942 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f791d93-bbaa-48ff-96d4-16cd24aba377 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:26.615476 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f791d93-bbaa-48ff-96d4-16cd24aba377 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:26.631780 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f791d93-bbaa-48ff-96d4-16cd24aba377 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:26.634532 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f791d93-bbaa-48ff-96d4-16cd24aba377 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:26.647929 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 965/1907] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:04:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:04:27.116272 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e23db-add9-46c2-aaae-8d970bb5e224 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:27.117699 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-670e23db-add9-46c2-aaae-8d970bb5e224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:27.132978 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e23db-add9-46c2-aaae-8d970bb5e224 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:27.139058 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e299c4c-e90d-4596-9c9c-0f38613808a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:27.141755 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e299c4c-e90d-4596-9c9c-0f38613808a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:27.141755 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e299c4c-e90d-4596-9c9c-0f38613808a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:27.141755 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e299c4c-e90d-4596-9c9c-0f38613808a9 admin admin] PATH_INFO: `/v3/projects/25b1d4822e9b421bad55479bd780ccee` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:27.141755 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e299c4c-e90d-4596-9c9c-0f38613808a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25b1d4822e9b421bad55479bd780ccee)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:27.143484 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e299c4c-e90d-4596-9c9c-0f38613808a9 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:27.271995 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5e299c4c-e90d-4596-9c9c-0f38613808a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25b1d4822e9b421bad55479bd780ccee', 'request_id': 'req-5e299c4c-e90d-4596-9c9c-0f38613808a9'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:27.279753 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5e299c4c-e90d-4596-9c9c-0f38613808a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25b1d4822e9b421bad55479bd780ccee', 'request_id': 'req-5e299c4c-e90d-4596-9c9c-0f38613808a9'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:27.285395 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5e299c4c-e90d-4596-9c9c-0f38613808a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25b1d4822e9b421bad55479bd780ccee', 'request_id': 'req-5e299c4c-e90d-4596-9c9c-0f38613808a9'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:27.396786 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5e299c4c-e90d-4596-9c9c-0f38613808a9 admin admin] The token cache is being invalidate because project 25b1d4822e9b421bad55479bd780ccee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:04:27.401972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5e299c4c-e90d-4596-9c9c-0f38613808a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:27.401972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5e299c4c-e90d-4596-9c9c-0f38613808a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:27.401972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 943/1908] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:04:27 2022] DELETE /identity/v3/projects/25b1d4822e9b421bad55479bd780ccee => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:27.405851 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c074093-8b79-4a27-bb54-adbc319b892f None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:27.406378 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c074093-8b79-4a27-bb54-adbc319b892f None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:27.406378 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c074093-8b79-4a27-bb54-adbc319b892f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:27.427066 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-0c074093-8b79-4a27-bb54-adbc319b892f None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:27.428297 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-0c074093-8b79-4a27-bb54-adbc319b892f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:27.492268 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 966/1909] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:27.500738 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2ec06-1758-4f07-9e2f-b5c2a0e2564f None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:27.501018 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2ec06-1758-4f07-9e2f-b5c2a0e2564f None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:27.501291 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2ec06-1758-4f07-9e2f-b5c2a0e2564f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:27.522349 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-14c2ec06-1758-4f07-9e2f-b5c2a0e2564f None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:27.523600 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-14c2ec06-1758-4f07-9e2f-b5c2a0e2564f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:27.532126 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 944/1910] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:04:27.536660 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c074093-8b79-4a27-bb54-adbc319b892f None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:27.546997 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c074093-8b79-4a27-bb54-adbc319b892f None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:27.553043 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a492477-109d-4659-9343-94867ed49ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:27.553847 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a492477-109d-4659-9343-94867ed49ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:27.554135 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a492477-109d-4659-9343-94867ed49ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:27.554421 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a492477-109d-4659-9343-94867ed49ed5 admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:27.555087 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a492477-109d-4659-9343-94867ed49ed5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:04:27.555392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a492477-109d-4659-9343-94867ed49ed5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:27.557573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a492477-109d-4659-9343-94867ed49ed5 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:27.575945 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 967/1911] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:04:27.578156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2ec06-1758-4f07-9e2f-b5c2a0e2564f None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:27.587688 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2ec06-1758-4f07-9e2f-b5c2a0e2564f None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:27.593667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4fecb2-827a-450a-abe4-9f183e07e2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:27.594354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fecb2-827a-450a-abe4-9f183e07e2b2 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:27.594672 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fecb2-827a-450a-abe4-9f183e07e2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:27.594940 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fecb2-827a-450a-abe4-9f183e07e2b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:27.595578 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4fecb2-827a-450a-abe4-9f183e07e2b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:27.597465 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4fecb2-827a-450a-abe4-9f183e07e2b2 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:27.727601 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 945/1912] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:04:27 2022] POST /identity/v3/projects => generated 381 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:04:27.729326 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a492477-109d-4659-9343-94867ed49ed5 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:27.741940 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a492477-109d-4659-9343-94867ed49ed5 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:27.763298 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:27.804876 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac26b85-fd2a-45ec-9021-3bb493abdd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:27.805704 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac26b85-fd2a-45ec-9021-3bb493abdd82 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:27.806130 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac26b85-fd2a-45ec-9021-3bb493abdd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:27.806440 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac26b85-fd2a-45ec-9021-3bb493abdd82 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:27.807644 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac26b85-fd2a-45ec-9021-3bb493abdd82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:27.809589 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac26b85-fd2a-45ec-9021-3bb493abdd82 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:27.894043 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 968/1913] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:04:27 2022] POST /identity/v3/users => generated 291 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:04:27.898133 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4fecb2-827a-450a-abe4-9f183e07e2b2 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:27.909618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4fecb2-827a-450a-abe4-9f183e07e2b2 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:27.914425 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bf6b32-88b8-477f-a17b-35c0ae6f4b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:27.915047 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf6b32-88b8-477f-a17b-35c0ae6f4b2f admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:27.915320 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf6b32-88b8-477f-a17b-35c0ae6f4b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:27.915613 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf6b32-88b8-477f-a17b-35c0ae6f4b2f admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:27.916255 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bf6b32-88b8-477f-a17b-35c0ae6f4b2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:27.917945 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bf6b32-88b8-477f-a17b-35c0ae6f4b2f admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:27.920361 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:27.928813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 946/1914] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:04:27.933316 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac26b85-fd2a-45ec-9021-3bb493abdd82 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:27.943830 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac26b85-fd2a-45ec-9021-3bb493abdd82 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:27.949322 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2c648-8912-44f3-b7c6-6291b7eadab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:27.949979 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2c648-8912-44f3-b7c6-6291b7eadab7 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:27.950254 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2c648-8912-44f3-b7c6-6291b7eadab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:27.950554 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2c648-8912-44f3-b7c6-6291b7eadab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:27.951206 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e2c648-8912-44f3-b7c6-6291b7eadab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:27.952853 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e2c648-8912-44f3-b7c6-6291b7eadab7 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:27.965823 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 969/1915] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:04:27.970221 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bf6b32-88b8-477f-a17b-35c0ae6f4b2f admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:27.981018 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bf6b32-88b8-477f-a17b-35c0ae6f4b2f admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:27.986334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25b57c-4ec8-4494-af3d-631991bc3c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:27.987115 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25b57c-4ec8-4494-af3d-631991bc3c9e admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:27.987399 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25b57c-4ec8-4494-af3d-631991bc3c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:27.987678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25b57c-4ec8-4494-af3d-631991bc3c9e admin admin] PATH_INFO: `/v3/projects/b9e701ae69b244d2993f92b44437a716/users/bccf1c23c68a4987882eedad6ec5c106/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:28.004847 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff25b57c-4ec8-4494-af3d-631991bc3c9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9e701ae69b244d2993f92b44437a716, user_id=bccf1c23c68a4987882eedad6ec5c106, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:28.007958 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff25b57c-4ec8-4494-af3d-631991bc3c9e admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:28.071536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 947/1916] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:04:27 2022] PUT /identity/v3/projects/b9e701ae69b244d2993f92b44437a716/users/bccf1c23c68a4987882eedad6ec5c106/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:28.080840 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802a689-9f46-4b10-b850-5a96378bb77d None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:28.081128 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802a689-9f46-4b10-b850-5a96378bb77d None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:28.081427 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802a689-9f46-4b10-b850-5a96378bb77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:28.104895 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-4802a689-9f46-4b10-b850-5a96378bb77d None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:28.106172 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-4802a689-9f46-4b10-b850-5a96378bb77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:28.171228 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 970/1917] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:28.180845 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25b57c-4ec8-4494-af3d-631991bc3c9e admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:28.183147 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-ff25b57c-4ec8-4494-af3d-631991bc3c9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:28.241089 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25b57c-4ec8-4494-af3d-631991bc3c9e admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:28.245901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9421436-74ca-4b77-b884-d201415b011c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:28.246582 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9421436-74ca-4b77-b884-d201415b011c service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:28.246862 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9421436-74ca-4b77-b884-d201415b011c service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:28.247133 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9421436-74ca-4b77-b884-d201415b011c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:28.256693 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9421436-74ca-4b77-b884-d201415b011c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:28.258131 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9421436-74ca-4b77-b884-d201415b011c service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:28.268885 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 948/1918] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:04:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:04:29.494151 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080a395-c7f0-402c-b71c-84a02e8e5443 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:29.494786 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080a395-c7f0-402c-b71c-84a02e8e5443 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:29.496176 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080a395-c7f0-402c-b71c-84a02e8e5443 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:29.499559 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 971/1919] 10.222.0.16 () {54 vars in 921 bytes} [Wed Aug 31 11:04:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:04:29.511087 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e021539-83a6-484c-91d6-2897ac967118 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:29.511644 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e021539-83a6-484c-91d6-2897ac967118 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:29.511644 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e021539-83a6-484c-91d6-2897ac967118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:29.533276 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-3e021539-83a6-484c-91d6-2897ac967118 None None] MFA Rules not processed for user `f11971a594014d86968c3356d11d42f7`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:29.534520 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-3e021539-83a6-484c-91d6-2897ac967118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:29.607757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 949/1920] 10.222.0.16 () {58 vars in 1018 bytes} [Wed Aug 31 11:04:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:29.630317 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080a395-c7f0-402c-b71c-84a02e8e5443 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:29.632140 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-e080a395-c7f0-402c-b71c-84a02e8e5443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:29.657554 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080a395-c7f0-402c-b71c-84a02e8e5443 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:29.666611 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a1988-c81c-497a-83de-f4b2f5842f6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:29.667552 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a1988-c81c-497a-83de-f4b2f5842f6b service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:29.667831 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a1988-c81c-497a-83de-f4b2f5842f6b service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:29.668104 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a1988-c81c-497a-83de-f4b2f5842f6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:29.678991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061a1988-c81c-497a-83de-f4b2f5842f6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:29.686639 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061a1988-c81c-497a-83de-f4b2f5842f6b service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:29.694484 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 972/1921] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:04:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:04:29.889716 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b1445-4cfc-4dd9-a3ff-1572b4ee45b4 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:29.890042 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b1445-4cfc-4dd9-a3ff-1572b4ee45b4 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:29.890682 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b1445-4cfc-4dd9-a3ff-1572b4ee45b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:29.920667 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a1988-c81c-497a-83de-f4b2f5842f6b service nova] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:29.920878 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-061a1988-c81c-497a-83de-f4b2f5842f6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:29.941188 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a1988-c81c-497a-83de-f4b2f5842f6b service nova] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:29.942846 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-8a4b1445-4cfc-4dd9-a3ff-1572b4ee45b4 None None] MFA Rules not processed for user `bccf1c23c68a4987882eedad6ec5c106`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:29.943904 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-8a4b1445-4cfc-4dd9-a3ff-1572b4ee45b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:29.953207 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d99817-cc27-46a3-86bb-14995a94edc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:29.954470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d99817-cc27-46a3-86bb-14995a94edc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:29.955055 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d99817-cc27-46a3-86bb-14995a94edc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:29.955546 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d99817-cc27-46a3-86bb-14995a94edc9 admin admin] PATH_INFO: `/v3/users/e1f8b7d5f65b4d53bdf3077f80206496` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:29.957087 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d99817-cc27-46a3-86bb-14995a94edc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1f8b7d5f65b4d53bdf3077f80206496)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:29.959342 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d99817-cc27-46a3-86bb-14995a94edc9 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:30.018336 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:30.028018 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 950/1922] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:30.035920 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b1445-4cfc-4dd9-a3ff-1572b4ee45b4 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:30.040822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-8a4b1445-4cfc-4dd9-a3ff-1572b4ee45b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:30.122982 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b1445-4cfc-4dd9-a3ff-1572b4ee45b4 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:30.127067 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f1ae3f-791d-43f2-a2b0-f70e4ddf6175 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:30.127788 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1ae3f-791d-43f2-a2b0-f70e4ddf6175 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:30.132642 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1ae3f-791d-43f2-a2b0-f70e4ddf6175 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:30.132642 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1ae3f-791d-43f2-a2b0-f70e4ddf6175 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:30.138174 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f1ae3f-791d-43f2-a2b0-f70e4ddf6175 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:30.139722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f1ae3f-791d-43f2-a2b0-f70e4ddf6175 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:30.151127 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 951/1923] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:04:30 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:04:30.240585 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-11d99817-cc27-46a3-86bb-14995a94edc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1f8b7d5f65b4d53bdf3077f80206496'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:30.253808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-11d99817-cc27-46a3-86bb-14995a94edc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1f8b7d5f65b4d53bdf3077f80206496'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:30.261084 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-11d99817-cc27-46a3-86bb-14995a94edc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1f8b7d5f65b4d53bdf3077f80206496'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:30.390942 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-11d99817-cc27-46a3-86bb-14995a94edc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1f8b7d5f65b4d53bdf3077f80206496'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:30.391758 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-11d99817-cc27-46a3-86bb-14995a94edc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1f8b7d5f65b4d53bdf3077f80206496'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:30.398012 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:30.406840 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 973/1924] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:04:29 2022] DELETE /identity/v3/users/e1f8b7d5f65b4d53bdf3077f80206496 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:30.408637 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22681355-8e7f-42d8-a740-3d80ec272ae2 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:30.408931 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22681355-8e7f-42d8-a740-3d80ec272ae2 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:30.409206 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22681355-8e7f-42d8-a740-3d80ec272ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:30.429317 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-22681355-8e7f-42d8-a740-3d80ec272ae2 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:30.430588 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-22681355-8e7f-42d8-a740-3d80ec272ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:30.495739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 952/1925] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:30.509353 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d99817-cc27-46a3-86bb-14995a94edc9 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:30.510947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-11d99817-cc27-46a3-86bb-14995a94edc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:30.571945 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d99817-cc27-46a3-86bb-14995a94edc9 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:30.575750 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b8c64-9f0f-48df-abc4-14663562385f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:30.576359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b8c64-9f0f-48df-abc4-14663562385f service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:30.576635 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b8c64-9f0f-48df-abc4-14663562385f service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:30.576904 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b8c64-9f0f-48df-abc4-14663562385f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:30.586359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b8c64-9f0f-48df-abc4-14663562385f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:30.587814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b8c64-9f0f-48df-abc4-14663562385f service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:30.597756 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 974/1926] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:04:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:04:31.214287 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22681355-8e7f-42d8-a740-3d80ec272ae2 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:31.216048 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-22681355-8e7f-42d8-a740-3d80ec272ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:31.234196 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22681355-8e7f-42d8-a740-3d80ec272ae2 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:31.240816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa6659-fcc0-4862-bb1d-893908465f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:31.247318 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa6659-fcc0-4862-bb1d-893908465f92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:31.247318 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa6659-fcc0-4862-bb1d-893908465f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:31.247318 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa6659-fcc0-4862-bb1d-893908465f92 admin admin] PATH_INFO: `/v3/projects/3088580f9e9149849dd00e609cee8f18` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:31.253855 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daa6659-fcc0-4862-bb1d-893908465f92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3088580f9e9149849dd00e609cee8f18)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:31.265962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daa6659-fcc0-4862-bb1d-893908465f92 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:31.283922 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b8c64-9f0f-48df-abc4-14663562385f service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:31.285587 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-dc2b8c64-9f0f-48df-abc4-14663562385f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:31.314572 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b8c64-9f0f-48df-abc4-14663562385f service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:31.329139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae77674-23c6-44a5-927e-3d8987891a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:31.336572 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae77674-23c6-44a5-927e-3d8987891a35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:31.336572 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae77674-23c6-44a5-927e-3d8987891a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:31.336572 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae77674-23c6-44a5-927e-3d8987891a35 admin admin] PATH_INFO: `/v3/users/6c7b814a997d4809b0aa9080596e6e83` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:31.338694 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae77674-23c6-44a5-927e-3d8987891a35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c7b814a997d4809b0aa9080596e6e83)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:31.340866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae77674-23c6-44a5-927e-3d8987891a35 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:31.497345 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-9daa6659-fcc0-4862-bb1d-893908465f92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3088580f9e9149849dd00e609cee8f18', 'request_id': 'req-9daa6659-fcc0-4862-bb1d-893908465f92'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:31.504893 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-9daa6659-fcc0-4862-bb1d-893908465f92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3088580f9e9149849dd00e609cee8f18', 'request_id': 'req-9daa6659-fcc0-4862-bb1d-893908465f92'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:31.512977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-9daa6659-fcc0-4862-bb1d-893908465f92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3088580f9e9149849dd00e609cee8f18', 'request_id': 'req-9daa6659-fcc0-4862-bb1d-893908465f92'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:31.654446 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-9daa6659-fcc0-4862-bb1d-893908465f92 admin admin] The token cache is being invalidate because project 3088580f9e9149849dd00e609cee8f18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:04:31.654446 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-9daa6659-fcc0-4862-bb1d-893908465f92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:31.654446 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-9daa6659-fcc0-4862-bb1d-893908465f92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:31.654823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 953/1927] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:04:31 2022] DELETE /identity/v3/projects/3088580f9e9149849dd00e609cee8f18 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:31.660453 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-2ae77674-23c6-44a5-927e-3d8987891a35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c7b814a997d4809b0aa9080596e6e83'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:31.661099 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa6659-fcc0-4862-bb1d-893908465f92 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:31.662001 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-9daa6659-fcc0-4862-bb1d-893908465f92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:31.673626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-2ae77674-23c6-44a5-927e-3d8987891a35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c7b814a997d4809b0aa9080596e6e83'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:31.681330 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-2ae77674-23c6-44a5-927e-3d8987891a35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c7b814a997d4809b0aa9080596e6e83'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:31.695605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:31.734896 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa6659-fcc0-4862-bb1d-893908465f92 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:31.739811 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44ddb2-98f6-444c-b3e4-858b449bf902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:31.740457 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44ddb2-98f6-444c-b3e4-858b449bf902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:31.740737 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44ddb2-98f6-444c-b3e4-858b449bf902 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:31.741033 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44ddb2-98f6-444c-b3e4-858b449bf902 admin admin] PATH_INFO: `/v3/users/01b5c25be2774354b691275d25bdd840` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:31.742243 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44ddb2-98f6-444c-b3e4-858b449bf902 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01b5c25be2774354b691275d25bdd840)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:31.745063 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44ddb2-98f6-444c-b3e4-858b449bf902 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:31.757756 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-2ae77674-23c6-44a5-927e-3d8987891a35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c7b814a997d4809b0aa9080596e6e83'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:31.758245 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-2ae77674-23c6-44a5-927e-3d8987891a35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c7b814a997d4809b0aa9080596e6e83'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:31.765053 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 975/1928] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:04:31 2022] DELETE /identity/v3/users/6c7b814a997d4809b0aa9080596e6e83 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:31.769773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec8bae6-7b68-4404-a24c-82ba77772faf None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:31.770065 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec8bae6-7b68-4404-a24c-82ba77772faf None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:31.770336 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec8bae6-7b68-4404-a24c-82ba77772faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:31.789615 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-2ec8bae6-7b68-4404-a24c-82ba77772faf None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:31.790879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2ec8bae6-7b68-4404-a24c-82ba77772faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:31.849328 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 976/1929] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:31.859879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec8bae6-7b68-4404-a24c-82ba77772faf None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:31.862026 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2ec8bae6-7b68-4404-a24c-82ba77772faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:31.880808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:31.936423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec8bae6-7b68-4404-a24c-82ba77772faf None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:31.939743 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f74afc2-0fb6-4f58-bb73-afa6defbd305 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:31.940356 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74afc2-0fb6-4f58-bb73-afa6defbd305 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:31.940634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74afc2-0fb6-4f58-bb73-afa6defbd305 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:31.940904 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74afc2-0fb6-4f58-bb73-afa6defbd305 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:31.951685 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f74afc2-0fb6-4f58-bb73-afa6defbd305 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:31.953562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f74afc2-0fb6-4f58-bb73-afa6defbd305 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:31.964709 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 977/1930] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:04:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:04:31.971087 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-bf44ddb2-98f6-444c-b3e4-858b449bf902 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01b5c25be2774354b691275d25bdd840'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:32.086127 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-bf44ddb2-98f6-444c-b3e4-858b449bf902 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01b5c25be2774354b691275d25bdd840'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:32.086763 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-bf44ddb2-98f6-444c-b3e4-858b449bf902 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01b5c25be2774354b691275d25bdd840'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:32.087370 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-bf44ddb2-98f6-444c-b3e4-858b449bf902 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01b5c25be2774354b691275d25bdd840'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:32.102228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-bf44ddb2-98f6-444c-b3e4-858b449bf902 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01b5c25be2774354b691275d25bdd840'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:32.120539 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 954/1931] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:04:31 2022] DELETE /identity/v3/users/01b5c25be2774354b691275d25bdd840 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:32.647621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f74afc2-0fb6-4f58-bb73-afa6defbd305 service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:32.649042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-7f74afc2-0fb6-4f58-bb73-afa6defbd305 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:32.662808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f74afc2-0fb6-4f58-bb73-afa6defbd305 service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:32.755228 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc156e-dc87-46cf-9ffd-dfb747ee3e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:32.755879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc156e-dc87-46cf-9ffd-dfb747ee3e60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:32.756156 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc156e-dc87-46cf-9ffd-dfb747ee3e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:32.756429 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc156e-dc87-46cf-9ffd-dfb747ee3e60 admin admin] PATH_INFO: `/v3/projects/04df4a9a868a4a66984212ee0bbd0a66` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:32.757602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc156e-dc87-46cf-9ffd-dfb747ee3e60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04df4a9a868a4a66984212ee0bbd0a66)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:32.759576 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc156e-dc87-46cf-9ffd-dfb747ee3e60 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:32.857922 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44ddb2-98f6-444c-b3e4-858b449bf902 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:32.866741 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-bf44ddb2-98f6-444c-b3e4-858b449bf902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:32.874993 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44ddb2-98f6-444c-b3e4-858b449bf902 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:32.880497 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4bfe74-d3c8-4a35-be08-fde342b35481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:32.881150 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4bfe74-d3c8-4a35-be08-fde342b35481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:32.881430 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4bfe74-d3c8-4a35-be08-fde342b35481 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:32.881895 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4bfe74-d3c8-4a35-be08-fde342b35481 admin admin] PATH_INFO: `/v3/projects/bc5978f3eb14436bac4c2617092de5a6` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:32.883123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4bfe74-d3c8-4a35-be08-fde342b35481 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc5978f3eb14436bac4c2617092de5a6)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:32.885212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4bfe74-d3c8-4a35-be08-fde342b35481 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:32.907777 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-6dbc156e-dc87-46cf-9ffd-dfb747ee3e60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04df4a9a868a4a66984212ee0bbd0a66', 'request_id': 'req-6dbc156e-dc87-46cf-9ffd-dfb747ee3e60'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:32.973471 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-6dbc156e-dc87-46cf-9ffd-dfb747ee3e60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04df4a9a868a4a66984212ee0bbd0a66', 'request_id': 'req-6dbc156e-dc87-46cf-9ffd-dfb747ee3e60'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:32.980588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-6dbc156e-dc87-46cf-9ffd-dfb747ee3e60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04df4a9a868a4a66984212ee0bbd0a66', 'request_id': 'req-6dbc156e-dc87-46cf-9ffd-dfb747ee3e60'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:32.992137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-6dbc156e-dc87-46cf-9ffd-dfb747ee3e60 admin admin] The token cache is being invalidate because project 04df4a9a868a4a66984212ee0bbd0a66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:04:32.992425 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-6dbc156e-dc87-46cf-9ffd-dfb747ee3e60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:32.993087 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-6dbc156e-dc87-46cf-9ffd-dfb747ee3e60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:32.994863 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 978/1932] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:04:32 2022] DELETE /identity/v3/projects/04df4a9a868a4a66984212ee0bbd0a66 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:32.998773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc156e-dc87-46cf-9ffd-dfb747ee3e60 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:33.000095 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-6dbc156e-dc87-46cf-9ffd-dfb747ee3e60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:33.021232 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-ef4bfe74-d3c8-4a35-be08-fde342b35481 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc5978f3eb14436bac4c2617092de5a6', 'request_id': 'req-ef4bfe74-d3c8-4a35-be08-fde342b35481'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:33.028717 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-ef4bfe74-d3c8-4a35-be08-fde342b35481 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc5978f3eb14436bac4c2617092de5a6', 'request_id': 'req-ef4bfe74-d3c8-4a35-be08-fde342b35481'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:33.035741 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-ef4bfe74-d3c8-4a35-be08-fde342b35481 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc5978f3eb14436bac4c2617092de5a6', 'request_id': 'req-ef4bfe74-d3c8-4a35-be08-fde342b35481'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:33.068939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc156e-dc87-46cf-9ffd-dfb747ee3e60 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:33.122449 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-ef4bfe74-d3c8-4a35-be08-fde342b35481 admin admin] The token cache is being invalidate because project bc5978f3eb14436bac4c2617092de5a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:04:33.122851 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-ef4bfe74-d3c8-4a35-be08-fde342b35481 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:33.123638 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-ef4bfe74-d3c8-4a35-be08-fde342b35481 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:33.125354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 955/1933] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:04:32 2022] DELETE /identity/v3/projects/bc5978f3eb14436bac4c2617092de5a6 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:33.129487 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a25f6-557f-4e5d-affd-ad4c534c84cf None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:33.138352 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a25f6-557f-4e5d-affd-ad4c534c84cf None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:33.138352 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a25f6-557f-4e5d-affd-ad4c534c84cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:33.138565 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef3a612-b7cd-4fe6-9b42-08718c25bc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:33.138565 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef3a612-b7cd-4fe6-9b42-08718c25bc7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:33.138565 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef3a612-b7cd-4fe6-9b42-08718c25bc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:33.138565 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef3a612-b7cd-4fe6-9b42-08718c25bc7c admin admin] PATH_INFO: `/v3/users/5066d7f1d7044ee6a71b63a526845c08` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:33.138854 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef3a612-b7cd-4fe6-9b42-08718c25bc7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5066d7f1d7044ee6a71b63a526845c08)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:33.139431 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef3a612-b7cd-4fe6-9b42-08718c25bc7c admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:33.150805 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-a55a25f6-557f-4e5d-affd-ad4c534c84cf None None] MFA Rules not processed for user `bccf1c23c68a4987882eedad6ec5c106`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:33.152106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a55a25f6-557f-4e5d-affd-ad4c534c84cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:33.173618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:33.212103 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 956/1934] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:33.212975 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a25f6-557f-4e5d-affd-ad4c534c84cf None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:33.214269 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a55a25f6-557f-4e5d-affd-ad4c534c84cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:33.229332 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a25f6-557f-4e5d-affd-ad4c534c84cf None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:33.234020 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53ee23-a458-4ffb-9679-3160d325361a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:33.234680 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53ee23-a458-4ffb-9679-3160d325361a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:33.234962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53ee23-a458-4ffb-9679-3160d325361a admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:33.235223 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53ee23-a458-4ffb-9679-3160d325361a admin admin] PATH_INFO: `/v3/users/528022f1b6e143b6933cfdcc874772c7` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:33.236345 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d53ee23-a458-4ffb-9679-3160d325361a admin admin] RBAC: Authorizing `identity:delete_user(user_id=528022f1b6e143b6933cfdcc874772c7)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:33.238236 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d53ee23-a458-4ffb-9679-3160d325361a admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:33.397330 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-5ef3a612-b7cd-4fe6-9b42-08718c25bc7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5066d7f1d7044ee6a71b63a526845c08'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:33.411743 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-5ef3a612-b7cd-4fe6-9b42-08718c25bc7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5066d7f1d7044ee6a71b63a526845c08'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:33.420776 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-5ef3a612-b7cd-4fe6-9b42-08718c25bc7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5066d7f1d7044ee6a71b63a526845c08'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:33.520711 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-5ef3a612-b7cd-4fe6-9b42-08718c25bc7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5066d7f1d7044ee6a71b63a526845c08'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:33.521585 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-5ef3a612-b7cd-4fe6-9b42-08718c25bc7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5066d7f1d7044ee6a71b63a526845c08'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:33.530608 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 979/1935] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:04:32 2022] DELETE /identity/v3/users/5066d7f1d7044ee6a71b63a526845c08 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:04:33.538452 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-4d53ee23-a458-4ffb-9679-3160d325361a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '528022f1b6e143b6933cfdcc874772c7'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:33.626467 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:33.653310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-4d53ee23-a458-4ffb-9679-3160d325361a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '528022f1b6e143b6933cfdcc874772c7'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:33.654327 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-4d53ee23-a458-4ffb-9679-3160d325361a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '528022f1b6e143b6933cfdcc874772c7'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:33.655102 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-4d53ee23-a458-4ffb-9679-3160d325361a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '528022f1b6e143b6933cfdcc874772c7'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:33.674821 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-4d53ee23-a458-4ffb-9679-3160d325361a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '528022f1b6e143b6933cfdcc874772c7'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:33.690813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 957/1936] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:04:33 2022] DELETE /identity/v3/users/528022f1b6e143b6933cfdcc874772c7 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:34.191902 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef3a612-b7cd-4fe6-9b42-08718c25bc7c admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:34.196563 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-5ef3a612-b7cd-4fe6-9b42-08718c25bc7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:34.259143 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53ee23-a458-4ffb-9679-3160d325361a admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:34.261498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-4d53ee23-a458-4ffb-9679-3160d325361a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:34.268349 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef3a612-b7cd-4fe6-9b42-08718c25bc7c admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:34.356080 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53ee23-a458-4ffb-9679-3160d325361a admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:34.359688 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec63d168-8bbc-43b1-ab72-983f63d576cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:34.360580 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63d168-8bbc-43b1-ab72-983f63d576cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:34.360803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a2c0b-3c4e-4458-a16a-1231d8440399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:34.360950 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63d168-8bbc-43b1-ab72-983f63d576cb admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:34.361326 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63d168-8bbc-43b1-ab72-983f63d576cb admin admin] PATH_INFO: `/v3/projects/65bf846fe87e4165ad3f6d46641cfcb9` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:34.361491 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a2c0b-3c4e-4458-a16a-1231d8440399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:34.361672 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a2c0b-3c4e-4458-a16a-1231d8440399 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:34.361866 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a2c0b-3c4e-4458-a16a-1231d8440399 admin admin] PATH_INFO: `/v3/projects/c8c29cff9bf245ee8c5709311ee06dec` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:34.362831 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec63d168-8bbc-43b1-ab72-983f63d576cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=65bf846fe87e4165ad3f6d46641cfcb9)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:34.363041 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a2c0b-3c4e-4458-a16a-1231d8440399 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8c29cff9bf245ee8c5709311ee06dec)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:34.364985 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a2c0b-3c4e-4458-a16a-1231d8440399 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:34.365250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec63d168-8bbc-43b1-ab72-983f63d576cb admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:34.507047 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-768a2c0b-3c4e-4458-a16a-1231d8440399 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8c29cff9bf245ee8c5709311ee06dec', 'request_id': 'req-768a2c0b-3c4e-4458-a16a-1231d8440399'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:34.513680 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-768a2c0b-3c4e-4458-a16a-1231d8440399 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8c29cff9bf245ee8c5709311ee06dec', 'request_id': 'req-768a2c0b-3c4e-4458-a16a-1231d8440399'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:34.519729 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-768a2c0b-3c4e-4458-a16a-1231d8440399 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8c29cff9bf245ee8c5709311ee06dec', 'request_id': 'req-768a2c0b-3c4e-4458-a16a-1231d8440399'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:34.520373 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ec63d168-8bbc-43b1-ab72-983f63d576cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65bf846fe87e4165ad3f6d46641cfcb9', 'request_id': 'req-ec63d168-8bbc-43b1-ab72-983f63d576cb'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:34.652868 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-768a2c0b-3c4e-4458-a16a-1231d8440399 admin admin] The token cache is being invalidate because project c8c29cff9bf245ee8c5709311ee06dec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:04:34.653163 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-768a2c0b-3c4e-4458-a16a-1231d8440399 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:34.653826 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-768a2c0b-3c4e-4458-a16a-1231d8440399 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:34.655589 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 958/1937] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:04:34 2022] DELETE /identity/v3/projects/c8c29cff9bf245ee8c5709311ee06dec => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:04:34.667226 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef299418-b08a-4f41-9b50-c3f200c1d374 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:34.667507 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef299418-b08a-4f41-9b50-c3f200c1d374 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:34.667793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef299418-b08a-4f41-9b50-c3f200c1d374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:34.689232 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-ef299418-b08a-4f41-9b50-c3f200c1d374 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:34.691237 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:34.696492 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-ef299418-b08a-4f41-9b50-c3f200c1d374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:34.715978 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ec63d168-8bbc-43b1-ab72-983f63d576cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65bf846fe87e4165ad3f6d46641cfcb9', 'request_id': 'req-ec63d168-8bbc-43b1-ab72-983f63d576cb'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:34.722263 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ec63d168-8bbc-43b1-ab72-983f63d576cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65bf846fe87e4165ad3f6d46641cfcb9', 'request_id': 'req-ec63d168-8bbc-43b1-ab72-983f63d576cb'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:34.734906 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ec63d168-8bbc-43b1-ab72-983f63d576cb admin admin] The token cache is being invalidate because project 65bf846fe87e4165ad3f6d46641cfcb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:04:34.735193 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ec63d168-8bbc-43b1-ab72-983f63d576cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:34.735844 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ec63d168-8bbc-43b1-ab72-983f63d576cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:34.737330 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 980/1938] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:04:34 2022] DELETE /identity/v3/projects/65bf846fe87e4165ad3f6d46641cfcb9 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:34.742173 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71bb93-c81a-44f5-868b-e28132195486 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:34.742574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71bb93-c81a-44f5-868b-e28132195486 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:34.743101 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71bb93-c81a-44f5-868b-e28132195486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:34.755235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 959/1939] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:04:34.763290 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaedb48-5a76-4059-8394-50ea12a3db55 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:34.763667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaedb48-5a76-4059-8394-50ea12a3db55 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:34.763847 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaedb48-5a76-4059-8394-50ea12a3db55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:34.764703 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-2f71bb93-c81a-44f5-868b-e28132195486 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:34.766009 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2f71bb93-c81a-44f5-868b-e28132195486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:34.774278 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 981/1940] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:34.783412 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b6d87-f60b-4adb-bbd9-92b8ebc857a0 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:34.783726 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b6d87-f60b-4adb-bbd9-92b8ebc857a0 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:34.784061 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b6d87-f60b-4adb-bbd9-92b8ebc857a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:34.785594 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-dcaedb48-5a76-4059-8394-50ea12a3db55 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:34.786858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-dcaedb48-5a76-4059-8394-50ea12a3db55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:34.796457 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 960/1941] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:04:34.802909 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaedb48-5a76-4059-8394-50ea12a3db55 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:34.806126 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-c89b6d87-f60b-4adb-bbd9-92b8ebc857a0 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:34.807924 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-c89b6d87-f60b-4adb-bbd9-92b8ebc857a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:34.813200 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaedb48-5a76-4059-8394-50ea12a3db55 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:34.816348 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 982/1942] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:34.819696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2bcffe-b46f-4125-8312-d0522d9ca756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:34.820259 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2bcffe-b46f-4125-8312-d0522d9ca756 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:34.820541 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2bcffe-b46f-4125-8312-d0522d9ca756 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:34.820645 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:34.820813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2bcffe-b46f-4125-8312-d0522d9ca756 admin admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:34.821888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b6d87-f60b-4adb-bbd9-92b8ebc857a0 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:34.823153 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2bcffe-b46f-4125-8312-d0522d9ca756 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:04:34.823546 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2bcffe-b46f-4125-8312-d0522d9ca756 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:34.825211 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2bcffe-b46f-4125-8312-d0522d9ca756 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:34.833054 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b6d87-f60b-4adb-bbd9-92b8ebc857a0 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:34.838710 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 961/1943] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:04:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:04:34.838904 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534c4e6-3fa7-465f-9fc0-3699ab238828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:34.839835 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534c4e6-3fa7-465f-9fc0-3699ab238828 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:34.840278 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534c4e6-3fa7-465f-9fc0-3699ab238828 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:34.840712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534c4e6-3fa7-465f-9fc0-3699ab238828 admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:34.841659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b534c4e6-3fa7-465f-9fc0-3699ab238828 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:04:34.842151 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b534c4e6-3fa7-465f-9fc0-3699ab238828 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:34.843804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2bcffe-b46f-4125-8312-d0522d9ca756 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:34.843922 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b534c4e6-3fa7-465f-9fc0-3699ab238828 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:34.855679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2bcffe-b46f-4125-8312-d0522d9ca756 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:34.861007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 983/1944] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:04:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:04:34.862142 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534c4e6-3fa7-465f-9fc0-3699ab238828 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:34.862669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86399d-9baa-4601-a54b-12657bb30810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:34.863539 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86399d-9baa-4601-a54b-12657bb30810 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:34.863830 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86399d-9baa-4601-a54b-12657bb30810 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:34.864190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86399d-9baa-4601-a54b-12657bb30810 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:34.864942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa86399d-9baa-4601-a54b-12657bb30810 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:34.866950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa86399d-9baa-4601-a54b-12657bb30810 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:34.873320 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534c4e6-3fa7-465f-9fc0-3699ab238828 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:34.878272 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd37fac8-4e0b-4229-9f41-ae4504a336f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:34.878953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37fac8-4e0b-4229-9f41-ae4504a336f3 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:34.879284 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37fac8-4e0b-4229-9f41-ae4504a336f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:34.879559 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37fac8-4e0b-4229-9f41-ae4504a336f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:34.880205 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd37fac8-4e0b-4229-9f41-ae4504a336f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:34.882172 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd37fac8-4e0b-4229-9f41-ae4504a336f3 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:34.996868 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 962/1945] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:04:34 2022] POST /identity/v3/projects => generated 371 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:04:34.999254 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86399d-9baa-4601-a54b-12657bb30810 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:35.024481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 984/1946] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:04:34 2022] POST /identity/v3/projects => generated 369 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:04:35.030151 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd37fac8-4e0b-4229-9f41-ae4504a336f3 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:35.081891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86399d-9baa-4601-a54b-12657bb30810 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:35.139465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd37fac8-4e0b-4229-9f41-ae4504a336f3 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:35.146413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973e663-8fbd-4550-b148-e619b7970ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:35.147038 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973e663-8fbd-4550-b148-e619b7970ecc admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:35.147318 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973e663-8fbd-4550-b148-e619b7970ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:35.147662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973e663-8fbd-4550-b148-e619b7970ecc admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:35.148274 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973e663-8fbd-4550-b148-e619b7970ecc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:35.148563 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44744ebb-7c96-4867-ac07-23b8ce760770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:35.149238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44744ebb-7c96-4867-ac07-23b8ce760770 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:35.149754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44744ebb-7c96-4867-ac07-23b8ce760770 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:35.150204 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44744ebb-7c96-4867-ac07-23b8ce760770 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:35.151095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973e663-8fbd-4550-b148-e619b7970ecc admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:35.151926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44744ebb-7c96-4867-ac07-23b8ce760770 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:35.153932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44744ebb-7c96-4867-ac07-23b8ce760770 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:35.249599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 985/1947] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:04:35 2022] POST /identity/v3/users => generated 285 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:04:35.252538 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 963/1948] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:04:34 2022] POST /identity/v3/users => generated 286 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:04:35.256246 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44744ebb-7c96-4867-ac07-23b8ce760770 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:35.259132 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973e663-8fbd-4550-b148-e619b7970ecc admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:35.268992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44744ebb-7c96-4867-ac07-23b8ce760770 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:35.270127 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973e663-8fbd-4550-b148-e619b7970ecc admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:35.274029 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9fb40e-987d-49bf-b309-f77bc14bc694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:35.275039 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9fb40e-987d-49bf-b309-f77bc14bc694 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:35.275705 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9fb40e-987d-49bf-b309-f77bc14bc694 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:35.276172 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9fb40e-987d-49bf-b309-f77bc14bc694 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:35.276728 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da02bf-91f2-4c39-b472-b562b46650fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:35.277303 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9fb40e-987d-49bf-b309-f77bc14bc694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:35.277649 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da02bf-91f2-4c39-b472-b562b46650fd admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:35.278089 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da02bf-91f2-4c39-b472-b562b46650fd admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:35.278651 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da02bf-91f2-4c39-b472-b562b46650fd admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:35.279653 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da02bf-91f2-4c39-b472-b562b46650fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:35.279941 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9fb40e-987d-49bf-b309-f77bc14bc694 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:35.282231 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da02bf-91f2-4c39-b472-b562b46650fd admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:35.295107 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 986/1949] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:04:35.296167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 964/1950] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:04:35.299113 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9fb40e-987d-49bf-b309-f77bc14bc694 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:35.300427 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da02bf-91f2-4c39-b472-b562b46650fd admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:35.312232 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da02bf-91f2-4c39-b472-b562b46650fd admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:35.312917 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9fb40e-987d-49bf-b309-f77bc14bc694 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:35.317175 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08149213-ccd9-4fd1-9930-b1932e4a3670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:35.317791 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08149213-ccd9-4fd1-9930-b1932e4a3670 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:35.318072 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08149213-ccd9-4fd1-9930-b1932e4a3670 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:35.318341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08149213-ccd9-4fd1-9930-b1932e4a3670 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:35.319030 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08149213-ccd9-4fd1-9930-b1932e4a3670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:35.319572 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147eb323-f607-4cc7-a296-0697e07668c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:35.320591 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147eb323-f607-4cc7-a296-0697e07668c3 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:35.320707 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08149213-ccd9-4fd1-9930-b1932e4a3670 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:35.321054 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147eb323-f607-4cc7-a296-0697e07668c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:35.321428 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147eb323-f607-4cc7-a296-0697e07668c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:35.322246 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147eb323-f607-4cc7-a296-0697e07668c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:35.324904 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147eb323-f607-4cc7-a296-0697e07668c3 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:35.332747 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 965/1951] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:04:35.337045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08149213-ccd9-4fd1-9930-b1932e4a3670 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:35.345904 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 987/1952] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:04:35.347109 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147eb323-f607-4cc7-a296-0697e07668c3 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:35.357399 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08149213-ccd9-4fd1-9930-b1932e4a3670 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:35.362857 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32dea1-c220-4aad-81ca-d0f6dd3bbfa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:35.363793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32dea1-c220-4aad-81ca-d0f6dd3bbfa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:35.364165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32dea1-c220-4aad-81ca-d0f6dd3bbfa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:35.364560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32dea1-c220-4aad-81ca-d0f6dd3bbfa2 admin admin] PATH_INFO: `/v3/projects/e20e3edeebd04d70be3f4116817593b3/users/8a1e00a787174dafba804a34a3eaf43c/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:35.370568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147eb323-f607-4cc7-a296-0697e07668c3 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:35.379170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ac597-7add-42ac-b6d3-9faf85b26add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:35.380108 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ac597-7add-42ac-b6d3-9faf85b26add admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:35.380549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ac597-7add-42ac-b6d3-9faf85b26add admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:35.380924 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ac597-7add-42ac-b6d3-9faf85b26add admin admin] PATH_INFO: `/v3/projects/24abc94d61054628a8fddd4fd8ad22d9/users/17a3394d5e8844bd96a2681a358cd87b/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:35.382728 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a32dea1-c220-4aad-81ca-d0f6dd3bbfa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e20e3edeebd04d70be3f4116817593b3, user_id=8a1e00a787174dafba804a34a3eaf43c, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:35.386169 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a32dea1-c220-4aad-81ca-d0f6dd3bbfa2 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:35.398914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ac597-7add-42ac-b6d3-9faf85b26add admin admin] RBAC: Authorizing `identity:create_grant(project_id=24abc94d61054628a8fddd4fd8ad22d9, user_id=17a3394d5e8844bd96a2681a358cd87b, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:35.401636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ac597-7add-42ac-b6d3-9faf85b26add admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:35.454208 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 966/1953] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:04:35 2022] PUT /identity/v3/projects/e20e3edeebd04d70be3f4116817593b3/users/8a1e00a787174dafba804a34a3eaf43c/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:35.467135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:35.469062 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8b1ba-5cbe-4923-8e6c-7f14a2fa2e08 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:35.469645 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8b1ba-5cbe-4923-8e6c-7f14a2fa2e08 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:35.469982 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8b1ba-5cbe-4923-8e6c-7f14a2fa2e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:35.474096 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 988/1954] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:04:35 2022] PUT /identity/v3/projects/24abc94d61054628a8fddd4fd8ad22d9/users/17a3394d5e8844bd96a2681a358cd87b/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:35.485959 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83eebf-bdd7-468f-8128-b61972de9eee None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:35.486238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83eebf-bdd7-468f-8128-b61972de9eee None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:35.486543 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83eebf-bdd7-468f-8128-b61972de9eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:35.501761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-7bb8b1ba-5cbe-4923-8e6c-7f14a2fa2e08 None None] MFA Rules not processed for user `8a1e00a787174dafba804a34a3eaf43c`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:35.503107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-7bb8b1ba-5cbe-4923-8e6c-7f14a2fa2e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:35.516421 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-0c83eebf-bdd7-468f-8128-b61972de9eee None None] MFA Rules not processed for user `17a3394d5e8844bd96a2681a358cd87b`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:35.518562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:35.522892 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-0c83eebf-bdd7-468f-8128-b61972de9eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:35.571529 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 967/1955] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:04:35.584836 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8b1ba-5cbe-4923-8e6c-7f14a2fa2e08 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:35.586316 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-7bb8b1ba-5cbe-4923-8e6c-7f14a2fa2e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:35.590529 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 989/1956] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:04:35.594968 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83eebf-bdd7-468f-8128-b61972de9eee None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:35.660379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83eebf-bdd7-468f-8128-b61972de9eee None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:35.664967 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030267c-36b2-4eea-babc-23fbaac247f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:35.665579 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030267c-36b2-4eea-babc-23fbaac247f1 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:35.665854 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030267c-36b2-4eea-babc-23fbaac247f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:35.666278 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030267c-36b2-4eea-babc-23fbaac247f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:35.666958 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0030267c-36b2-4eea-babc-23fbaac247f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:35.668946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0030267c-36b2-4eea-babc-23fbaac247f1 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:35.670372 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8b1ba-5cbe-4923-8e6c-7f14a2fa2e08 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:35.674047 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82d36e-5cd2-4a1c-8457-647959ec9358 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2e72c8e7b444e978bf99c6404febcd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:35.674795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82d36e-5cd2-4a1c-8457-647959ec9358 service glance] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:35.675069 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82d36e-5cd2-4a1c-8457-647959ec9358 service glance] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:35.675346 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82d36e-5cd2-4a1c-8457-647959ec9358 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:35.682619 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:35.685096 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca82d36e-5cd2-4a1c-8457-647959ec9358 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:35.686568 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca82d36e-5cd2-4a1c-8457-647959ec9358 service glance] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:35.696438 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 968/1957] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:04:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:04:35.740689 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 990/1958] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:04:35 2022] POST /identity/v3/projects => generated 367 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:04:35.745739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82d36e-5cd2-4a1c-8457-647959ec9358 service glance] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:35.766491 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82d36e-5cd2-4a1c-8457-647959ec9358 service glance] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:35.828049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c30a39-9722-432b-b962-c989c2fdd241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:35.828834 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c30a39-9722-432b-b962-c989c2fdd241 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:35.829355 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c30a39-9722-432b-b962-c989c2fdd241 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:35.829355 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c30a39-9722-432b-b962-c989c2fdd241 admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:35.830062 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c30a39-9722-432b-b962-c989c2fdd241 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:35.832105 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c30a39-9722-432b-b962-c989c2fdd241 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:35.921643 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 969/1959] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:04:35 2022] POST /identity/v3/users => generated 290 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:04:35.927116 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030267c-36b2-4eea-babc-23fbaac247f1 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:35.938698 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030267c-36b2-4eea-babc-23fbaac247f1 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:35.944638 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da974756-0957-4822-889b-e1083e789942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:35.945306 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da974756-0957-4822-889b-e1083e789942 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:35.945576 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da974756-0957-4822-889b-e1083e789942 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:35.945852 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da974756-0957-4822-889b-e1083e789942 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:35.946973 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da974756-0957-4822-889b-e1083e789942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:35.948279 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da974756-0957-4822-889b-e1083e789942 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:35.963762 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 991/1960] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:04:35.965748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c30a39-9722-432b-b962-c989c2fdd241 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:35.977259 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c30a39-9722-432b-b962-c989c2fdd241 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:35.983681 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03919af2-bfd2-45bb-b846-5bda39e824fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:35.984371 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03919af2-bfd2-45bb-b846-5bda39e824fd admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:35.984650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03919af2-bfd2-45bb-b846-5bda39e824fd admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:35.985079 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03919af2-bfd2-45bb-b846-5bda39e824fd admin admin] PATH_INFO: `/v3/projects/f508843d6d734f9b97bb9f7ed0229fc9/users/0d1133f97fd8457abacb884fc17d05fd/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:36.003242 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03919af2-bfd2-45bb-b846-5bda39e824fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f508843d6d734f9b97bb9f7ed0229fc9, user_id=0d1133f97fd8457abacb884fc17d05fd, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:36.006057 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03919af2-bfd2-45bb-b846-5bda39e824fd admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:36.102088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 970/1961] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:04:35 2022] PUT /identity/v3/projects/f508843d6d734f9b97bb9f7ed0229fc9/users/0d1133f97fd8457abacb884fc17d05fd/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:36.113680 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da974756-0957-4822-889b-e1083e789942 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:36.128902 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da974756-0957-4822-889b-e1083e789942 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:36.141386 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:36.220191 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09268a7e-b3bc-46ce-98b2-c2ec45ebeeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:36.220808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09268a7e-b3bc-46ce-98b2-c2ec45ebeeb6 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:36.221084 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09268a7e-b3bc-46ce-98b2-c2ec45ebeeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:36.221353 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09268a7e-b3bc-46ce-98b2-c2ec45ebeeb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:36.221999 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09268a7e-b3bc-46ce-98b2-c2ec45ebeeb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:36.223625 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09268a7e-b3bc-46ce-98b2-c2ec45ebeeb6 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:36.237537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 992/1962] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:04:36.238993 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03919af2-bfd2-45bb-b846-5bda39e824fd admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:36.251251 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03919af2-bfd2-45bb-b846-5bda39e824fd admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:36.255998 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f116a2-1286-49d0-a183-e315a14ee406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:36.256661 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f116a2-1286-49d0-a183-e315a14ee406 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:36.256933 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f116a2-1286-49d0-a183-e315a14ee406 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:36.257200 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f116a2-1286-49d0-a183-e315a14ee406 admin admin] PATH_INFO: `/v3/projects/f508843d6d734f9b97bb9f7ed0229fc9/users/0d1133f97fd8457abacb884fc17d05fd/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:36.261418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f116a2-1286-49d0-a183-e315a14ee406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f508843d6d734f9b97bb9f7ed0229fc9, user_id=0d1133f97fd8457abacb884fc17d05fd, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:36.262739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f116a2-1286-49d0-a183-e315a14ee406 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:36.274910 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 971/1963] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:04:36 2022] PUT /identity/v3/projects/f508843d6d734f9b97bb9f7ed0229fc9/users/0d1133f97fd8457abacb884fc17d05fd/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:04:36.286526 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf184a1-9fa7-4416-88f3-aec521cecb1c None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:36.287076 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf184a1-9fa7-4416-88f3-aec521cecb1c None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:36.287076 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf184a1-9fa7-4416-88f3-aec521cecb1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:36.320066 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-9bf184a1-9fa7-4416-88f3-aec521cecb1c None None] MFA Rules not processed for user `0d1133f97fd8457abacb884fc17d05fd`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:36.321430 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-9bf184a1-9fa7-4416-88f3-aec521cecb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:36.409584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 993/1964] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:36.430589 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f116a2-1286-49d0-a183-e315a14ee406 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:36.433318 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-69f116a2-1286-49d0-a183-e315a14ee406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:36.502319 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f116a2-1286-49d0-a183-e315a14ee406 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:36.504250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af82f9e-02b6-4603-8981-56f24f9b3524 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:36.504921 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af82f9e-02b6-4603-8981-56f24f9b3524 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:36.505204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af82f9e-02b6-4603-8981-56f24f9b3524 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:36.505473 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af82f9e-02b6-4603-8981-56f24f9b3524 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:36.518340 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af82f9e-02b6-4603-8981-56f24f9b3524 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:36.520406 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af82f9e-02b6-4603-8981-56f24f9b3524 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:36.535058 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 972/1965] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:04:36 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:04:36.654286 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf184a1-9fa7-4416-88f3-aec521cecb1c None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:36.655779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-9bf184a1-9fa7-4416-88f3-aec521cecb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:36.669420 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf184a1-9fa7-4416-88f3-aec521cecb1c None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:36.675547 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0ed2aa-c377-45ac-a2a6-b0288b725912 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:36.676152 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ed2aa-c377-45ac-a2a6-b0288b725912 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:36.676152 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ed2aa-c377-45ac-a2a6-b0288b725912 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:36.676386 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ed2aa-c377-45ac-a2a6-b0288b725912 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:36.740633 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0ed2aa-c377-45ac-a2a6-b0288b725912 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:36.742074 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0ed2aa-c377-45ac-a2a6-b0288b725912 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:36.751869 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 994/1966] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:04:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:04:42.439419 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af82f9e-02b6-4603-8981-56f24f9b3524 service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:42.517251 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:42.530827 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af82f9e-02b6-4603-8981-56f24f9b3524 service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:42.536561 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b47cb-7189-493e-b2f9-577b3c74e0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:42.537612 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b47cb-7189-493e-b2f9-577b3c74e0f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:42.538042 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b47cb-7189-493e-b2f9-577b3c74e0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:42.538229 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b47cb-7189-493e-b2f9-577b3c74e0f0 admin admin] PATH_INFO: `/v3/users/17a3394d5e8844bd96a2681a358cd87b` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:42.539975 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b47cb-7189-493e-b2f9-577b3c74e0f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17a3394d5e8844bd96a2681a358cd87b)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:42.542205 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b47cb-7189-493e-b2f9-577b3c74e0f0 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:42.777197 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2d6b47cb-7189-493e-b2f9-577b3c74e0f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17a3394d5e8844bd96a2681a358cd87b'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:42.874937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2d6b47cb-7189-493e-b2f9-577b3c74e0f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17a3394d5e8844bd96a2681a358cd87b'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:42.875850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2d6b47cb-7189-493e-b2f9-577b3c74e0f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17a3394d5e8844bd96a2681a358cd87b'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:42.876481 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2d6b47cb-7189-493e-b2f9-577b3c74e0f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17a3394d5e8844bd96a2681a358cd87b'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:42.907395 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2d6b47cb-7189-493e-b2f9-577b3c74e0f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17a3394d5e8844bd96a2681a358cd87b'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:42.923472 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 973/1967] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:04:42 2022] DELETE /identity/v3/users/17a3394d5e8844bd96a2681a358cd87b => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:42.927886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e676b-2f2a-43c1-9c34-7a7f5d475a23 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:42.929254 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e676b-2f2a-43c1-9c34-7a7f5d475a23 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:42.929254 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e676b-2f2a-43c1-9c34-7a7f5d475a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:42.949006 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-560e676b-2f2a-43c1-9c34-7a7f5d475a23 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:42.950208 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-560e676b-2f2a-43c1-9c34-7a7f5d475a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:42.977011 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:43.011318 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 995/1968] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:43.021364 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b47cb-7189-493e-b2f9-577b3c74e0f0 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:43.022938 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-2d6b47cb-7189-493e-b2f9-577b3c74e0f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:43.084854 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b47cb-7189-493e-b2f9-577b3c74e0f0 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:43.089215 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb20bcd-4cfd-421e-a2d7-d15bf416ba67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:43.089855 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb20bcd-4cfd-421e-a2d7-d15bf416ba67 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:43.090134 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb20bcd-4cfd-421e-a2d7-d15bf416ba67 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:43.091738 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb20bcd-4cfd-421e-a2d7-d15bf416ba67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:43.112625 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb20bcd-4cfd-421e-a2d7-d15bf416ba67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:43.114280 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb20bcd-4cfd-421e-a2d7-d15bf416ba67 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:43.130443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 974/1969] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:04:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:04:43.229134 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e676b-2f2a-43c1-9c34-7a7f5d475a23 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:43.230714 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-560e676b-2f2a-43c1-9c34-7a7f5d475a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:43.244988 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e676b-2f2a-43c1-9c34-7a7f5d475a23 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:43.251165 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd0ca6-834a-45a5-a206-9edecb05a19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:43.252017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd0ca6-834a-45a5-a206-9edecb05a19e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:43.252298 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd0ca6-834a-45a5-a206-9edecb05a19e admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:43.252647 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd0ca6-834a-45a5-a206-9edecb05a19e admin admin] PATH_INFO: `/v3/users/044355b41ef645b386d54e1955c0417c` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:43.253793 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bd0ca6-834a-45a5-a206-9edecb05a19e admin admin] RBAC: Authorizing `identity:delete_user(user_id=044355b41ef645b386d54e1955c0417c)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:43.256187 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bd0ca6-834a-45a5-a206-9edecb05a19e admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:43.434092 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-d9bd0ca6-834a-45a5-a206-9edecb05a19e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '044355b41ef645b386d54e1955c0417c'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:43.448394 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-d9bd0ca6-834a-45a5-a206-9edecb05a19e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '044355b41ef645b386d54e1955c0417c'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:43.457014 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-d9bd0ca6-834a-45a5-a206-9edecb05a19e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '044355b41ef645b386d54e1955c0417c'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:43.565325 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-d9bd0ca6-834a-45a5-a206-9edecb05a19e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '044355b41ef645b386d54e1955c0417c'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:43.565325 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-d9bd0ca6-834a-45a5-a206-9edecb05a19e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '044355b41ef645b386d54e1955c0417c'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:43.577475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 996/1970] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:04:43 2022] DELETE /identity/v3/users/044355b41ef645b386d54e1955c0417c => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:43.588842 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217efe0-1900-4f8c-993c-1a4caa924f05 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:43.589687 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217efe0-1900-4f8c-993c-1a4caa924f05 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:43.589964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217efe0-1900-4f8c-993c-1a4caa924f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:43.610609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-2217efe0-1900-4f8c-993c-1a4caa924f05 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:43.612112 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-2217efe0-1900-4f8c-993c-1a4caa924f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:43.697105 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd0ca6-834a-45a5-a206-9edecb05a19e admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:43.699013 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-d9bd0ca6-834a-45a5-a206-9edecb05a19e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:43.711921 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 975/1971] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:43.726619 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd0ca6-834a-45a5-a206-9edecb05a19e admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:43.731410 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2217efe0-1900-4f8c-993c-1a4caa924f05 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:43.734295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-2217efe0-1900-4f8c-993c-1a4caa924f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:43.746484 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6dd87e-3dd1-4212-95bf-138ae4127b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:43.747296 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6dd87e-3dd1-4212-95bf-138ae4127b81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:43.753705 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:43.755062 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6dd87e-3dd1-4212-95bf-138ae4127b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:43.756411 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6dd87e-3dd1-4212-95bf-138ae4127b81 admin admin] PATH_INFO: `/v3/projects/24abc94d61054628a8fddd4fd8ad22d9` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:43.758325 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6dd87e-3dd1-4212-95bf-138ae4127b81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24abc94d61054628a8fddd4fd8ad22d9)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:43.760468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6dd87e-3dd1-4212-95bf-138ae4127b81 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:43.841847 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2217efe0-1900-4f8c-993c-1a4caa924f05 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:43.845609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c0da3-7e78-47e0-a67d-7599109684b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:43.847618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c0da3-7e78-47e0-a67d-7599109684b1 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:43.847925 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c0da3-7e78-47e0-a67d-7599109684b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:43.848201 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c0da3-7e78-47e0-a67d-7599109684b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:43.861295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c0da3-7e78-47e0-a67d-7599109684b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:43.862986 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c0da3-7e78-47e0-a67d-7599109684b1 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:43.880470 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 976/1972] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:04:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:04:43.935912 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-be6dd87e-3dd1-4212-95bf-138ae4127b81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24abc94d61054628a8fddd4fd8ad22d9', 'request_id': 'req-be6dd87e-3dd1-4212-95bf-138ae4127b81'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:44.042702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-be6dd87e-3dd1-4212-95bf-138ae4127b81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24abc94d61054628a8fddd4fd8ad22d9', 'request_id': 'req-be6dd87e-3dd1-4212-95bf-138ae4127b81'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:44.050166 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-be6dd87e-3dd1-4212-95bf-138ae4127b81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24abc94d61054628a8fddd4fd8ad22d9', 'request_id': 'req-be6dd87e-3dd1-4212-95bf-138ae4127b81'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:44.077620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-be6dd87e-3dd1-4212-95bf-138ae4127b81 admin admin] The token cache is being invalidate because project 24abc94d61054628a8fddd4fd8ad22d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:04:44.077895 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-be6dd87e-3dd1-4212-95bf-138ae4127b81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:44.078811 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-be6dd87e-3dd1-4212-95bf-138ae4127b81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:44.080602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 997/1973] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:04:43 2022] DELETE /identity/v3/projects/24abc94d61054628a8fddd4fd8ad22d9 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:04:44.084893 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c0da3-7e78-47e0-a67d-7599109684b1 service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:44.096587 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c0da3-7e78-47e0-a67d-7599109684b1 service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:44.155077 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b08f9-7ee5-4466-8fba-2dca75fb1d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:44.156053 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b08f9-7ee5-4466-8fba-2dca75fb1d5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:44.156175 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b08f9-7ee5-4466-8fba-2dca75fb1d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:44.156391 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b08f9-7ee5-4466-8fba-2dca75fb1d5f admin admin] PATH_INFO: `/v3/users/0d1133f97fd8457abacb884fc17d05fd` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:44.157515 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b08f9-7ee5-4466-8fba-2dca75fb1d5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d1133f97fd8457abacb884fc17d05fd)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:44.159497 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b08f9-7ee5-4466-8fba-2dca75fb1d5f admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:44.372081 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-d47b08f9-7ee5-4466-8fba-2dca75fb1d5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d1133f97fd8457abacb884fc17d05fd'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:44.469970 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6dd87e-3dd1-4212-95bf-138ae4127b81 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:44.476014 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-be6dd87e-3dd1-4212-95bf-138ae4127b81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:44.494740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-d47b08f9-7ee5-4466-8fba-2dca75fb1d5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d1133f97fd8457abacb884fc17d05fd'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:44.496035 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-d47b08f9-7ee5-4466-8fba-2dca75fb1d5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d1133f97fd8457abacb884fc17d05fd'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:44.497160 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-d47b08f9-7ee5-4466-8fba-2dca75fb1d5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d1133f97fd8457abacb884fc17d05fd'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:44.526823 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6dd87e-3dd1-4212-95bf-138ae4127b81 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:44.528060 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-d47b08f9-7ee5-4466-8fba-2dca75fb1d5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d1133f97fd8457abacb884fc17d05fd'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:44.533604 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccec2e-f235-4dad-ad16-84c3996c6cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:44.534631 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccec2e-f235-4dad-ad16-84c3996c6cb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:44.534631 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccec2e-f235-4dad-ad16-84c3996c6cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:44.535532 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccec2e-f235-4dad-ad16-84c3996c6cb2 admin admin] PATH_INFO: `/v3/projects/125145a02a6b4b32959536cf97a3b0df` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:44.537821 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ccec2e-f235-4dad-ad16-84c3996c6cb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=125145a02a6b4b32959536cf97a3b0df)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:44.539889 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ccec2e-f235-4dad-ad16-84c3996c6cb2 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:44.544757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 977/1974] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:04:44 2022] DELETE /identity/v3/users/0d1133f97fd8457abacb884fc17d05fd => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:44.575151 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:44.676848 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-36ccec2e-f235-4dad-ad16-84c3996c6cb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '125145a02a6b4b32959536cf97a3b0df', 'request_id': 'req-36ccec2e-f235-4dad-ad16-84c3996c6cb2'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:44.807524 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-36ccec2e-f235-4dad-ad16-84c3996c6cb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '125145a02a6b4b32959536cf97a3b0df', 'request_id': 'req-36ccec2e-f235-4dad-ad16-84c3996c6cb2'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:44.823370 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-36ccec2e-f235-4dad-ad16-84c3996c6cb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '125145a02a6b4b32959536cf97a3b0df', 'request_id': 'req-36ccec2e-f235-4dad-ad16-84c3996c6cb2'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:44.844862 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-36ccec2e-f235-4dad-ad16-84c3996c6cb2 admin admin] The token cache is being invalidate because project 125145a02a6b4b32959536cf97a3b0df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:04:44.858292 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-36ccec2e-f235-4dad-ad16-84c3996c6cb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:44.859996 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-36ccec2e-f235-4dad-ad16-84c3996c6cb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:44.871918 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 998/1975] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:04:44 2022] DELETE /identity/v3/projects/125145a02a6b4b32959536cf97a3b0df => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:44.882466 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b08f9-7ee5-4466-8fba-2dca75fb1d5f admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:44.882466 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-d47b08f9-7ee5-4466-8fba-2dca75fb1d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:44.923274 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:44.963596 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b08f9-7ee5-4466-8fba-2dca75fb1d5f admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:45.078102 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf4a0c3-176c-444e-b71c-d0cc20d3f4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:45.078102 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4a0c3-176c-444e-b71c-d0cc20d3f4d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:45.078102 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4a0c3-176c-444e-b71c-d0cc20d3f4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:45.078102 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4a0c3-176c-444e-b71c-d0cc20d3f4d0 admin admin] PATH_INFO: `/v3/users/49221778883942e0907f265bdf146495` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:45.078741 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf4a0c3-176c-444e-b71c-d0cc20d3f4d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49221778883942e0907f265bdf146495)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:45.078741 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf4a0c3-176c-444e-b71c-d0cc20d3f4d0 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:45.323664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-fdf4a0c3-176c-444e-b71c-d0cc20d3f4d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49221778883942e0907f265bdf146495'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:45.394262 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-fdf4a0c3-176c-444e-b71c-d0cc20d3f4d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49221778883942e0907f265bdf146495'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:45.395171 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccec2e-f235-4dad-ad16-84c3996c6cb2 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:45.395646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-fdf4a0c3-176c-444e-b71c-d0cc20d3f4d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49221778883942e0907f265bdf146495'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:45.396559 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-fdf4a0c3-176c-444e-b71c-d0cc20d3f4d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49221778883942e0907f265bdf146495'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:45.397311 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-36ccec2e-f235-4dad-ad16-84c3996c6cb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:45.414530 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-fdf4a0c3-176c-444e-b71c-d0cc20d3f4d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49221778883942e0907f265bdf146495'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:45.433602 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 978/1976] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:04:44 2022] DELETE /identity/v3/users/49221778883942e0907f265bdf146495 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:45.438972 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccec2e-f235-4dad-ad16-84c3996c6cb2 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:45.507226 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d6854-7587-4e3c-9017-d516cf6b1fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:45.507824 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d6854-7587-4e3c-9017-d516cf6b1fb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:45.508101 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d6854-7587-4e3c-9017-d516cf6b1fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:45.508365 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d6854-7587-4e3c-9017-d516cf6b1fb3 admin admin] PATH_INFO: `/v3/projects/f508843d6d734f9b97bb9f7ed0229fc9` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:45.510212 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d6854-7587-4e3c-9017-d516cf6b1fb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f508843d6d734f9b97bb9f7ed0229fc9)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:45.512152 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d6854-7587-4e3c-9017-d516cf6b1fb3 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:45.646712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e61d6854-7587-4e3c-9017-d516cf6b1fb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f508843d6d734f9b97bb9f7ed0229fc9', 'request_id': 'req-e61d6854-7587-4e3c-9017-d516cf6b1fb3'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:45.746801 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e61d6854-7587-4e3c-9017-d516cf6b1fb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f508843d6d734f9b97bb9f7ed0229fc9', 'request_id': 'req-e61d6854-7587-4e3c-9017-d516cf6b1fb3'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:45.753032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e61d6854-7587-4e3c-9017-d516cf6b1fb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f508843d6d734f9b97bb9f7ed0229fc9', 'request_id': 'req-e61d6854-7587-4e3c-9017-d516cf6b1fb3'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:45.764615 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e61d6854-7587-4e3c-9017-d516cf6b1fb3 admin admin] The token cache is being invalidate because project f508843d6d734f9b97bb9f7ed0229fc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:04:45.764906 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e61d6854-7587-4e3c-9017-d516cf6b1fb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:45.765615 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e61d6854-7587-4e3c-9017-d516cf6b1fb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:45.767171 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 999/1977] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:04:45 2022] DELETE /identity/v3/projects/f508843d6d734f9b97bb9f7ed0229fc9 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:45.772163 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a37b11-59db-4591-86e0-1ddb2181d2f9 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:45.772450 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a37b11-59db-4591-86e0-1ddb2181d2f9 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:45.772724 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a37b11-59db-4591-86e0-1ddb2181d2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:45.796993 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-38a37b11-59db-4591-86e0-1ddb2181d2f9 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:45.798199 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-38a37b11-59db-4591-86e0-1ddb2181d2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:45.857877 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 979/1978] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:45.866421 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8345df-439b-4686-b0f8-56939c03d296 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:45.868565 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8345df-439b-4686-b0f8-56939c03d296 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:45.868565 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8345df-439b-4686-b0f8-56939c03d296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:45.872714 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a37b11-59db-4591-86e0-1ddb2181d2f9 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:45.874369 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-38a37b11-59db-4591-86e0-1ddb2181d2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:45.888078 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a37b11-59db-4591-86e0-1ddb2181d2f9 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:45.897510 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de09675-7ee9-492c-84d4-a0aa37669a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:45.897510 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de09675-7ee9-492c-84d4-a0aa37669a51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:45.897510 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de09675-7ee9-492c-84d4-a0aa37669a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:45.897510 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de09675-7ee9-492c-84d4-a0aa37669a51 admin admin] PATH_INFO: `/v3/projects/e02d08c7af744469a1d8a389b18896a3` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:45.897897 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de09675-7ee9-492c-84d4-a0aa37669a51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e02d08c7af744469a1d8a389b18896a3)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:45.898880 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de09675-7ee9-492c-84d4-a0aa37669a51 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:45.929686 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-ee8345df-439b-4686-b0f8-56939c03d296 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:45.930973 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-ee8345df-439b-4686-b0f8-56939c03d296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:45.939974 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1000/1979] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:45.945769 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8345df-439b-4686-b0f8-56939c03d296 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:45.956937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8345df-439b-4686-b0f8-56939c03d296 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:45.963898 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7a624-84e6-4663-8820-33288ae30910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:45.964525 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7a624-84e6-4663-8820-33288ae30910 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:45.964798 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7a624-84e6-4663-8820-33288ae30910 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:45.965460 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7a624-84e6-4663-8820-33288ae30910 admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:45.965707 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf7a624-84e6-4663-8820-33288ae30910 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:04:45.966006 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf7a624-84e6-4663-8820-33288ae30910 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:45.967715 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf7a624-84e6-4663-8820-33288ae30910 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:45.984398 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1001/1980] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:04:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:04:45.985976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7a624-84e6-4663-8820-33288ae30910 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:45.998765 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7a624-84e6-4663-8820-33288ae30910 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:46.004332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a2487-1c73-4b7d-b9e9-e357ae867cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:46.005286 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a2487-1c73-4b7d-b9e9-e357ae867cf0 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:46.005564 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a2487-1c73-4b7d-b9e9-e357ae867cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:46.005896 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a2487-1c73-4b7d-b9e9-e357ae867cf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:46.006595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a2487-1c73-4b7d-b9e9-e357ae867cf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:46.008562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a2487-1c73-4b7d-b9e9-e357ae867cf0 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:46.042598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-7de09675-7ee9-492c-84d4-a0aa37669a51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e02d08c7af744469a1d8a389b18896a3', 'request_id': 'req-7de09675-7ee9-492c-84d4-a0aa37669a51'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:46.049746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-7de09675-7ee9-492c-84d4-a0aa37669a51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e02d08c7af744469a1d8a389b18896a3', 'request_id': 'req-7de09675-7ee9-492c-84d4-a0aa37669a51'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:46.057597 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-7de09675-7ee9-492c-84d4-a0aa37669a51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e02d08c7af744469a1d8a389b18896a3', 'request_id': 'req-7de09675-7ee9-492c-84d4-a0aa37669a51'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:46.113073 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1002/1981] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:04:45 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:04:46.117774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a2487-1c73-4b7d-b9e9-e357ae867cf0 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:46.144837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-7de09675-7ee9-492c-84d4-a0aa37669a51 admin admin] The token cache is being invalidate because project e02d08c7af744469a1d8a389b18896a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:04:46.144837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-7de09675-7ee9-492c-84d4-a0aa37669a51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:46.146473 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-7de09675-7ee9-492c-84d4-a0aa37669a51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:46.147412 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 980/1982] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:04:45 2022] DELETE /identity/v3/projects/e02d08c7af744469a1d8a389b18896a3 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:46.153385 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f4c88-8d21-463b-aea8-bb38d5e3e148 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:46.153767 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f4c88-8d21-463b-aea8-bb38d5e3e148 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:46.154119 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f4c88-8d21-463b-aea8-bb38d5e3e148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:46.177502 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-109f4c88-8d21-463b-aea8-bb38d5e3e148 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:46.179099 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-109f4c88-8d21-463b-aea8-bb38d5e3e148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:46.195035 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:46.229363 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a2487-1c73-4b7d-b9e9-e357ae867cf0 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:46.235047 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2ca36-736c-4879-977d-a1c46f7b64b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:46.235672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2ca36-736c-4879-977d-a1c46f7b64b7 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:46.235972 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2ca36-736c-4879-977d-a1c46f7b64b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:46.236449 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2ca36-736c-4879-977d-a1c46f7b64b7 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:46.237003 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced2ca36-736c-4879-977d-a1c46f7b64b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:46.238969 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced2ca36-736c-4879-977d-a1c46f7b64b7 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:46.253157 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 981/1983] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:46.261844 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac9eba-83d9-40c3-adac-16fc084b030d None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:46.262266 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac9eba-83d9-40c3-adac-16fc084b030d None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:46.262585 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac9eba-83d9-40c3-adac-16fc084b030d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:46.287199 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-20ac9eba-83d9-40c3-adac-16fc084b030d None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:46.288839 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-20ac9eba-83d9-40c3-adac-16fc084b030d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:46.302546 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 982/1984] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:04:46.308287 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac9eba-83d9-40c3-adac-16fc084b030d None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:46.324559 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1003/1985] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:04:46 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:04:46.346426 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac9eba-83d9-40c3-adac-16fc084b030d None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:46.347273 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2ca36-736c-4879-977d-a1c46f7b64b7 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:46.347273 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-ced2ca36-736c-4879-977d-a1c46f7b64b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:46.354904 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2ca36-736c-4879-977d-a1c46f7b64b7 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:46.356640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65c161-d12d-4965-ad8d-75cd1bca6633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:46.357472 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65c161-d12d-4965-ad8d-75cd1bca6633 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:46.357955 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65c161-d12d-4965-ad8d-75cd1bca6633 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:46.358226 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65c161-d12d-4965-ad8d-75cd1bca6633 admin admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:46.358936 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee65c161-d12d-4965-ad8d-75cd1bca6633 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:04:46.359233 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee65c161-d12d-4965-ad8d-75cd1bca6633 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:46.360907 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee65c161-d12d-4965-ad8d-75cd1bca6633 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:46.361413 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe057f36-df8a-4635-be3e-0a15fd1ce8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:46.362259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe057f36-df8a-4635-be3e-0a15fd1ce8cc admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:46.362756 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe057f36-df8a-4635-be3e-0a15fd1ce8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:46.363033 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe057f36-df8a-4635-be3e-0a15fd1ce8cc admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:46.363698 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe057f36-df8a-4635-be3e-0a15fd1ce8cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:46.365828 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe057f36-df8a-4635-be3e-0a15fd1ce8cc admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:46.375691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 983/1986] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:04:46.380470 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65c161-d12d-4965-ad8d-75cd1bca6633 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:46.381839 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1004/1987] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:04:46.386197 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe057f36-df8a-4635-be3e-0a15fd1ce8cc admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:46.391247 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65c161-d12d-4965-ad8d-75cd1bca6633 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:46.395927 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe057f36-df8a-4635-be3e-0a15fd1ce8cc admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:46.396385 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d437fed-d22f-4d21-97e6-5a53dd43a842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:46.396945 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d437fed-d22f-4d21-97e6-5a53dd43a842 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:46.397317 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d437fed-d22f-4d21-97e6-5a53dd43a842 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:46.397647 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d437fed-d22f-4d21-97e6-5a53dd43a842 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:46.398284 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d437fed-d22f-4d21-97e6-5a53dd43a842 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:46.400599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d437fed-d22f-4d21-97e6-5a53dd43a842 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:46.401512 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d119a9d-ea4b-4f01-aa79-dc8e6a7b075d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:46.402112 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d119a9d-ea4b-4f01-aa79-dc8e6a7b075d admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:46.404251 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d119a9d-ea4b-4f01-aa79-dc8e6a7b075d admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:46.404251 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d119a9d-ea4b-4f01-aa79-dc8e6a7b075d admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:46.404251 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d119a9d-ea4b-4f01-aa79-dc8e6a7b075d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:46.405074 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d119a9d-ea4b-4f01-aa79-dc8e6a7b075d admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:46.416711 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1005/1988] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:04:46.420876 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d119a9d-ea4b-4f01-aa79-dc8e6a7b075d admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:46.432094 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d119a9d-ea4b-4f01-aa79-dc8e6a7b075d admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:46.437241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009cf0a-e45b-4a6c-9a9e-bcb450eccccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:46.437902 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009cf0a-e45b-4a6c-9a9e-bcb450eccccf admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:46.438183 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009cf0a-e45b-4a6c-9a9e-bcb450eccccf admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:46.438562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009cf0a-e45b-4a6c-9a9e-bcb450eccccf admin admin] PATH_INFO: `/v3/projects/32f667fd1dbb4f6d8b4756abd70ee99d/users/59bd4cdf60094d2bb134e2852aec7959/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:46.455480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9009cf0a-e45b-4a6c-9a9e-bcb450eccccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=32f667fd1dbb4f6d8b4756abd70ee99d, user_id=59bd4cdf60094d2bb134e2852aec7959, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:46.458563 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9009cf0a-e45b-4a6c-9a9e-bcb450eccccf admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:46.514087 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 984/1989] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:04:46 2022] POST /identity/v3/projects => generated 401 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:04:46.515212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d437fed-d22f-4d21-97e6-5a53dd43a842 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:46.540231 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1006/1990] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:04:46 2022] PUT /identity/v3/projects/32f667fd1dbb4f6d8b4756abd70ee99d/users/59bd4cdf60094d2bb134e2852aec7959/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:04:46.554349 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b49d1-f02c-4da3-8d5c-91604548bc52 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:46.554715 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b49d1-f02c-4da3-8d5c-91604548bc52 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:46.554941 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b49d1-f02c-4da3-8d5c-91604548bc52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:46.588674 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d437fed-d22f-4d21-97e6-5a53dd43a842 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:46.595586 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-898b49d1-f02c-4da3-8d5c-91604548bc52 None None] MFA Rules not processed for user `59bd4cdf60094d2bb134e2852aec7959`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:46.596835 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-898b49d1-f02c-4da3-8d5c-91604548bc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:46.637847 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:46.669841 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0daf7e6-af2d-4d55-ab72-1715ae58643d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:46.670961 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0daf7e6-af2d-4d55-ab72-1715ae58643d admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:46.671402 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0daf7e6-af2d-4d55-ab72-1715ae58643d admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:46.671840 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0daf7e6-af2d-4d55-ab72-1715ae58643d admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:46.672488 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0daf7e6-af2d-4d55-ab72-1715ae58643d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:46.674550 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0daf7e6-af2d-4d55-ab72-1715ae58643d admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:46.684382 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1007/1991] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:04:46.689004 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b49d1-f02c-4da3-8d5c-91604548bc52 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:46.700865 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b49d1-f02c-4da3-8d5c-91604548bc52 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:46.705848 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca7e791-d599-4c1e-981c-d44f49635ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:46.706593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7e791-d599-4c1e-981c-d44f49635ac2 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:46.706872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7e791-d599-4c1e-981c-d44f49635ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:46.707151 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7e791-d599-4c1e-981c-d44f49635ac2 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:46.707804 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca7e791-d599-4c1e-981c-d44f49635ac2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:46.709737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca7e791-d599-4c1e-981c-d44f49635ac2 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:46.737940 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 985/1992] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:04:46 2022] POST /identity/v3/users => generated 301 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:04:46.739524 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0daf7e6-af2d-4d55-ab72-1715ae58643d admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:46.749613 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0daf7e6-af2d-4d55-ab72-1715ae58643d admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:46.754631 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0998008-3a23-4e35-a600-9624dbc15c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:46.755207 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0998008-3a23-4e35-a600-9624dbc15c54 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:46.755485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0998008-3a23-4e35-a600-9624dbc15c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:46.755756 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0998008-3a23-4e35-a600-9624dbc15c54 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:46.756404 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0998008-3a23-4e35-a600-9624dbc15c54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:46.758035 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0998008-3a23-4e35-a600-9624dbc15c54 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:46.768870 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 986/1993] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:04:46.772899 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0998008-3a23-4e35-a600-9624dbc15c54 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:46.785819 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1008/1994] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:04:46 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:04:46.794334 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca7e791-d599-4c1e-981c-d44f49635ac2 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:46.795640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0998008-3a23-4e35-a600-9624dbc15c54 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:46.865880 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afef14-1be4-4d12-825f-096292fd1118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:46.867180 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afef14-1be4-4d12-825f-096292fd1118 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:46.867180 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afef14-1be4-4d12-825f-096292fd1118 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:46.867180 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afef14-1be4-4d12-825f-096292fd1118 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:46.867899 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24afef14-1be4-4d12-825f-096292fd1118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:46.869804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24afef14-1be4-4d12-825f-096292fd1118 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:46.870763 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca7e791-d599-4c1e-981c-d44f49635ac2 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:46.875809 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306fd2d-1de7-4395-b914-75a6f8553fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:46.876430 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306fd2d-1de7-4395-b914-75a6f8553fca admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:46.876714 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306fd2d-1de7-4395-b914-75a6f8553fca admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:46.876985 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306fd2d-1de7-4395-b914-75a6f8553fca admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:46.877637 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306fd2d-1de7-4395-b914-75a6f8553fca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:46.879631 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306fd2d-1de7-4395-b914-75a6f8553fca admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:46.881064 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 987/1995] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:04:46.885248 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afef14-1be4-4d12-825f-096292fd1118 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:46.896138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afef14-1be4-4d12-825f-096292fd1118 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:46.902056 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f7f7d-6b3b-4131-9bb0-61b11814b508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:46.903223 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f7f7d-6b3b-4131-9bb0-61b11814b508 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:46.903572 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f7f7d-6b3b-4131-9bb0-61b11814b508 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:46.903912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f7f7d-6b3b-4131-9bb0-61b11814b508 admin admin] PATH_INFO: `/v3/projects/b946f75bbbe54338a65eab944a697ed6/users/d4bf531772584b9689ec2fdfe7122336/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:46.923485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f7f7d-6b3b-4131-9bb0-61b11814b508 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b946f75bbbe54338a65eab944a697ed6, user_id=d4bf531772584b9689ec2fdfe7122336, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:46.926461 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f7f7d-6b3b-4131-9bb0-61b11814b508 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:46.990654 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1009/1996] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:04:46 2022] POST /identity/v3/users => generated 299 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:04:46.994821 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306fd2d-1de7-4395-b914-75a6f8553fca admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:47.004531 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306fd2d-1de7-4395-b914-75a6f8553fca admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:47.009176 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676a0c5-db6b-4508-b77b-d37885ede01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:47.009869 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676a0c5-db6b-4508-b77b-d37885ede01f admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:47.010373 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676a0c5-db6b-4508-b77b-d37885ede01f admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:47.010697 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676a0c5-db6b-4508-b77b-d37885ede01f admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:47.011367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676a0c5-db6b-4508-b77b-d37885ede01f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:47.014164 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676a0c5-db6b-4508-b77b-d37885ede01f admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:47.019849 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 988/1997] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:04:46 2022] PUT /identity/v3/projects/b946f75bbbe54338a65eab944a697ed6/users/d4bf531772584b9689ec2fdfe7122336/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:04:47.024531 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1010/1998] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:04:47.029435 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f7f7d-6b3b-4131-9bb0-61b11814b508 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:47.031511 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c32b7-4fad-4a5c-8b12-e108de2ef306 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:47.031762 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c32b7-4fad-4a5c-8b12-e108de2ef306 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:47.032096 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c32b7-4fad-4a5c-8b12-e108de2ef306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:47.061569 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-365c32b7-4fad-4a5c-8b12-e108de2ef306 None None] MFA Rules not processed for user `d4bf531772584b9689ec2fdfe7122336`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:47.062864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-365c32b7-4fad-4a5c-8b12-e108de2ef306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:47.096090 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f7f7d-6b3b-4131-9bb0-61b11814b508 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:47.101565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91294a6b-2c29-40d5-98f0-5605d59918d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:47.103846 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91294a6b-2c29-40d5-98f0-5605d59918d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:47.103846 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91294a6b-2c29-40d5-98f0-5605d59918d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:47.103846 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91294a6b-2c29-40d5-98f0-5605d59918d8 admin admin] PATH_INFO: `/v3/projects/5eef473f80cd4fbbae8935b112129ec1/users/83144d6ae2854c39b198dc75c3a9075b/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:47.119487 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91294a6b-2c29-40d5-98f0-5605d59918d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eef473f80cd4fbbae8935b112129ec1, user_id=83144d6ae2854c39b198dc75c3a9075b, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:47.129982 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91294a6b-2c29-40d5-98f0-5605d59918d8 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:47.139198 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1011/1999] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:47 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:04:47.143634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c32b7-4fad-4a5c-8b12-e108de2ef306 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:47.155283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c32b7-4fad-4a5c-8b12-e108de2ef306 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:47.159944 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39ef67-d088-49ff-baf1-c716f5ca8356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:47.160553 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39ef67-d088-49ff-baf1-c716f5ca8356 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:47.160834 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39ef67-d088-49ff-baf1-c716f5ca8356 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:47.161171 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39ef67-d088-49ff-baf1-c716f5ca8356 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:47.161815 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb39ef67-d088-49ff-baf1-c716f5ca8356 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:47.163745 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb39ef67-d088-49ff-baf1-c716f5ca8356 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:47.207354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 989/2000] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:04:47 2022] PUT /identity/v3/projects/5eef473f80cd4fbbae8935b112129ec1/users/83144d6ae2854c39b198dc75c3a9075b/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:47.212137 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91294a6b-2c29-40d5-98f0-5605d59918d8 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:47.293293 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1012/2001] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:04:47 2022] POST /identity/v3/projects => generated 401 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:04:47.295068 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39ef67-d088-49ff-baf1-c716f5ca8356 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:47.314095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91294a6b-2c29-40d5-98f0-5605d59918d8 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:47.408868 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39ef67-d088-49ff-baf1-c716f5ca8356 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:47.409822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5646c-c428-400b-a0d4-a64e02c4e3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:47.409822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5646c-c428-400b-a0d4-a64e02c4e3de admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:47.409822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5646c-c428-400b-a0d4-a64e02c4e3de admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:47.409822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5646c-c428-400b-a0d4-a64e02c4e3de admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:47.410359 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5646c-c428-400b-a0d4-a64e02c4e3de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:47.410359 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5646c-c428-400b-a0d4-a64e02c4e3de admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:47.414067 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8001c-dc53-4ef8-b6bc-c92f40fb9d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:47.415179 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8001c-dc53-4ef8-b6bc-c92f40fb9d81 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:47.415955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8001c-dc53-4ef8-b6bc-c92f40fb9d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:47.417453 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8001c-dc53-4ef8-b6bc-c92f40fb9d81 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:47.418574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8001c-dc53-4ef8-b6bc-c92f40fb9d81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:47.420951 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8001c-dc53-4ef8-b6bc-c92f40fb9d81 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:47.448261 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 990/2002] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:04:47.451937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5646c-c428-400b-a0d4-a64e02c4e3de admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:47.462945 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5646c-c428-400b-a0d4-a64e02c4e3de admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:47.464077 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:47.468269 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec7645-af64-4a1b-87bc-0692c6282bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:47.469070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec7645-af64-4a1b-87bc-0692c6282bfb admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:47.469643 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec7645-af64-4a1b-87bc-0692c6282bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:47.470009 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec7645-af64-4a1b-87bc-0692c6282bfb admin admin] PATH_INFO: `/v3/projects/5eef473f80cd4fbbae8935b112129ec1/users/83144d6ae2854c39b198dc75c3a9075b/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:47.473311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec7645-af64-4a1b-87bc-0692c6282bfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eef473f80cd4fbbae8935b112129ec1, user_id=83144d6ae2854c39b198dc75c3a9075b, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:47.477498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec7645-af64-4a1b-87bc-0692c6282bfb admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:47.490249 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 991/2003] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:04:47 2022] PUT /identity/v3/projects/5eef473f80cd4fbbae8935b112129ec1/users/83144d6ae2854c39b198dc75c3a9075b/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:47.512031 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48786ae-7b7d-4009-a309-3a578e90e090 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:47.513069 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48786ae-7b7d-4009-a309-3a578e90e090 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:47.513352 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48786ae-7b7d-4009-a309-3a578e90e090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:47.531383 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1013/2004] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:04:47 2022] POST /identity/v3/users => generated 307 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:04:47.532911 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8001c-dc53-4ef8-b6bc-c92f40fb9d81 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:47.547588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8001c-dc53-4ef8-b6bc-c92f40fb9d81 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:47.553177 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-b48786ae-7b7d-4009-a309-3a578e90e090 None None] MFA Rules not processed for user `83144d6ae2854c39b198dc75c3a9075b`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:47.554489 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-b48786ae-7b7d-4009-a309-3a578e90e090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:47.619559 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:47.628606 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab4e08-f94a-419f-a16e-db9b4ee239da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:47.629639 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab4e08-f94a-419f-a16e-db9b4ee239da admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:47.630138 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab4e08-f94a-419f-a16e-db9b4ee239da admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:47.630551 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab4e08-f94a-419f-a16e-db9b4ee239da admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:47.631437 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ab4e08-f94a-419f-a16e-db9b4ee239da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:47.633223 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ab4e08-f94a-419f-a16e-db9b4ee239da admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:47.653066 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1014/2005] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:04:47.653261 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 992/2006] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:47 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:04:47.655639 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab4e08-f94a-419f-a16e-db9b4ee239da admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:47.663767 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab4e08-f94a-419f-a16e-db9b4ee239da admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:47.666761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48786ae-7b7d-4009-a309-3a578e90e090 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:47.668898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-b48786ae-7b7d-4009-a309-3a578e90e090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:47.669510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7ba85-6a84-41cb-ba72-2088de938fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:47.670343 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7ba85-6a84-41cb-ba72-2088de938fcc admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:47.670796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7ba85-6a84-41cb-ba72-2088de938fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:47.671235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7ba85-6a84-41cb-ba72-2088de938fcc admin admin] PATH_INFO: `/v3/projects/056112a0e10049e3b726f28b6709eecd/users/a930a76ad7764359b6c4424e8b725235/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:47.694440 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd7ba85-6a84-41cb-ba72-2088de938fcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=056112a0e10049e3b726f28b6709eecd, user_id=a930a76ad7764359b6c4424e8b725235, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:47.714682 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd7ba85-6a84-41cb-ba72-2088de938fcc admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:47.770054 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48786ae-7b7d-4009-a309-3a578e90e090 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:47.774996 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef80ce9-b3ae-43ff-9c8a-3da4478913c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:47.775902 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef80ce9-b3ae-43ff-9c8a-3da4478913c9 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:47.776225 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef80ce9-b3ae-43ff-9c8a-3da4478913c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:47.776587 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef80ce9-b3ae-43ff-9c8a-3da4478913c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:47.791891 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1015/2007] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:04:47 2022] PUT /identity/v3/projects/056112a0e10049e3b726f28b6709eecd/users/a930a76ad7764359b6c4424e8b725235/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:04:47.795954 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7ba85-6a84-41cb-ba72-2088de938fcc admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:47.861484 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef80ce9-b3ae-43ff-9c8a-3da4478913c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:47.863013 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef80ce9-b3ae-43ff-9c8a-3da4478913c9 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:47.874245 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 993/2008] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:04:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:04:47.877891 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7ba85-6a84-41cb-ba72-2088de938fcc admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:47.883199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68cc02-4bd8-4c3f-aa8e-5dee42e1b778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:47.883813 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68cc02-4bd8-4c3f-aa8e-5dee42e1b778 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:47.884255 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68cc02-4bd8-4c3f-aa8e-5dee42e1b778 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:47.884678 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68cc02-4bd8-4c3f-aa8e-5dee42e1b778 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:47.885235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68cc02-4bd8-4c3f-aa8e-5dee42e1b778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:47.886857 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68cc02-4bd8-4c3f-aa8e-5dee42e1b778 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:47.897159 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1016/2009] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:04:47.901588 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef80ce9-b3ae-43ff-9c8a-3da4478913c9 service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:47.913127 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef80ce9-b3ae-43ff-9c8a-3da4478913c9 service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:47.918626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16607b6-44d4-425f-bcde-d840144b7d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:47.919305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16607b6-44d4-425f-bcde-d840144b7d12 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:47.919577 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16607b6-44d4-425f-bcde-d840144b7d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:47.923975 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16607b6-44d4-425f-bcde-d840144b7d12 admin admin] PATH_INFO: `/v3/projects/056112a0e10049e3b726f28b6709eecd/users/a930a76ad7764359b6c4424e8b725235/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:47.923975 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16607b6-44d4-425f-bcde-d840144b7d12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=056112a0e10049e3b726f28b6709eecd, user_id=a930a76ad7764359b6c4424e8b725235, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:47.924439 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16607b6-44d4-425f-bcde-d840144b7d12 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:47.936349 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 994/2010] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:04:47 2022] PUT /identity/v3/projects/056112a0e10049e3b726f28b6709eecd/users/a930a76ad7764359b6c4424e8b725235/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:47.943385 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:47.947914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87751a15-7414-409b-bbef-46f2dc9afc26 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:47.948201 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87751a15-7414-409b-bbef-46f2dc9afc26 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:47.948477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87751a15-7414-409b-bbef-46f2dc9afc26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:47.982010 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-87751a15-7414-409b-bbef-46f2dc9afc26 None None] MFA Rules not processed for user `a930a76ad7764359b6c4424e8b725235`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:47.983799 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-87751a15-7414-409b-bbef-46f2dc9afc26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:48.063168 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1017/2011] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:47 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:48.070087 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16607b6-44d4-425f-bcde-d840144b7d12 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:48.128790 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16607b6-44d4-425f-bcde-d840144b7d12 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:48.132436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf0f04-c607-4d61-83fc-7929b07d38e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:48.133038 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf0f04-c607-4d61-83fc-7929b07d38e4 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:48.133312 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf0f04-c607-4d61-83fc-7929b07d38e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:48.133581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf0f04-c607-4d61-83fc-7929b07d38e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:48.143641 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bf0f04-c607-4d61-83fc-7929b07d38e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:48.145073 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bf0f04-c607-4d61-83fc-7929b07d38e4 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:48.155433 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 995/2012] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:04:48 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:04:48.392908 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87751a15-7414-409b-bbef-46f2dc9afc26 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:48.405608 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87751a15-7414-409b-bbef-46f2dc9afc26 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:48.409466 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c90c841-cbe0-4464-b618-afe2b33c4daa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:48.410371 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90c841-cbe0-4464-b618-afe2b33c4daa service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:48.410371 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90c841-cbe0-4464-b618-afe2b33c4daa service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:48.410760 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90c841-cbe0-4464-b618-afe2b33c4daa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:48.475963 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c90c841-cbe0-4464-b618-afe2b33c4daa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:48.477625 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c90c841-cbe0-4464-b618-afe2b33c4daa service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:48.501478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1018/2013] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:04:48 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:04:48.528426 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:48.597260 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:49.215351 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:54.945873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf0f04-c607-4d61-83fc-7929b07d38e4 service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:54.988022 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:55.033032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf0f04-c607-4d61-83fc-7929b07d38e4 service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:55.039974 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328b42f-faa8-4e0c-b43f-dbe05a78ffc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:55.040571 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328b42f-faa8-4e0c-b43f-dbe05a78ffc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:55.040885 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328b42f-faa8-4e0c-b43f-dbe05a78ffc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:55.041159 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328b42f-faa8-4e0c-b43f-dbe05a78ffc1 admin admin] PATH_INFO: `/v3/users/d4bf531772584b9689ec2fdfe7122336` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:55.042540 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2328b42f-faa8-4e0c-b43f-dbe05a78ffc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4bf531772584b9689ec2fdfe7122336)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:55.044557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2328b42f-faa8-4e0c-b43f-dbe05a78ffc1 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:55.244250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2328b42f-faa8-4e0c-b43f-dbe05a78ffc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4bf531772584b9689ec2fdfe7122336'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:55.384714 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2328b42f-faa8-4e0c-b43f-dbe05a78ffc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4bf531772584b9689ec2fdfe7122336'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:55.385863 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2328b42f-faa8-4e0c-b43f-dbe05a78ffc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4bf531772584b9689ec2fdfe7122336'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:55.386655 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2328b42f-faa8-4e0c-b43f-dbe05a78ffc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4bf531772584b9689ec2fdfe7122336'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:55.402764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2328b42f-faa8-4e0c-b43f-dbe05a78ffc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4bf531772584b9689ec2fdfe7122336'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:55.417202 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 996/2014] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:04:54 2022] DELETE /identity/v3/users/d4bf531772584b9689ec2fdfe7122336 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:55.421988 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b64cc2-678c-4e70-9dc8-632d17e6c13a None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:55.422544 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b64cc2-678c-4e70-9dc8-632d17e6c13a None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:55.422647 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b64cc2-678c-4e70-9dc8-632d17e6c13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:55.443288 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-b3b64cc2-678c-4e70-9dc8-632d17e6c13a None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:55.444533 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-b3b64cc2-678c-4e70-9dc8-632d17e6c13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:55.509202 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1019/2015] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:55.519019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328b42f-faa8-4e0c-b43f-dbe05a78ffc1 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:55.520490 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-2328b42f-faa8-4e0c-b43f-dbe05a78ffc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:55.582885 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328b42f-faa8-4e0c-b43f-dbe05a78ffc1 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:55.586537 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac521dec-c44c-4f54-8205-5464352d758d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:55.587144 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac521dec-c44c-4f54-8205-5464352d758d service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:55.587417 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac521dec-c44c-4f54-8205-5464352d758d service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:55.587683 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac521dec-c44c-4f54-8205-5464352d758d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:55.596775 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac521dec-c44c-4f54-8205-5464352d758d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:55.598186 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac521dec-c44c-4f54-8205-5464352d758d service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:55.608623 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 997/2016] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:04:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:04:56.155337 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b64cc2-678c-4e70-9dc8-632d17e6c13a None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:56.156849 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-b3b64cc2-678c-4e70-9dc8-632d17e6c13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:56.183423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b64cc2-678c-4e70-9dc8-632d17e6c13a None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:56.207456 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d596b55-bf2c-4083-9b69-c161b53b96ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:56.208320 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d596b55-bf2c-4083-9b69-c161b53b96ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:56.208694 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d596b55-bf2c-4083-9b69-c161b53b96ef admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:56.209254 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d596b55-bf2c-4083-9b69-c161b53b96ef admin admin] PATH_INFO: `/v3/projects/b946f75bbbe54338a65eab944a697ed6` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:56.211021 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d596b55-bf2c-4083-9b69-c161b53b96ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=b946f75bbbe54338a65eab944a697ed6)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:56.215305 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d596b55-bf2c-4083-9b69-c161b53b96ef admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:56.367071 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-1d596b55-bf2c-4083-9b69-c161b53b96ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b946f75bbbe54338a65eab944a697ed6', 'request_id': 'req-1d596b55-bf2c-4083-9b69-c161b53b96ef'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:56.465317 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-1d596b55-bf2c-4083-9b69-c161b53b96ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b946f75bbbe54338a65eab944a697ed6', 'request_id': 'req-1d596b55-bf2c-4083-9b69-c161b53b96ef'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:56.474082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-1d596b55-bf2c-4083-9b69-c161b53b96ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b946f75bbbe54338a65eab944a697ed6', 'request_id': 'req-1d596b55-bf2c-4083-9b69-c161b53b96ef'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:56.486411 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:56.493232 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-1d596b55-bf2c-4083-9b69-c161b53b96ef admin admin] The token cache is being invalidate because project b946f75bbbe54338a65eab944a697ed6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:04:56.493776 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-1d596b55-bf2c-4083-9b69-c161b53b96ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:56.495621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-1d596b55-bf2c-4083-9b69-c161b53b96ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:56.498882 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1020/2017] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:04:56 2022] DELETE /identity/v3/projects/b946f75bbbe54338a65eab944a697ed6 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:56.502304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac521dec-c44c-4f54-8205-5464352d758d service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:56.514471 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac521dec-c44c-4f54-8205-5464352d758d service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:56.589398 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:56.605492 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783bc12-82fd-4fb6-98e6-8c3920398996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:56.606229 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783bc12-82fd-4fb6-98e6-8c3920398996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:56.606607 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783bc12-82fd-4fb6-98e6-8c3920398996 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:56.606948 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783bc12-82fd-4fb6-98e6-8c3920398996 admin admin] PATH_INFO: `/v3/users/a930a76ad7764359b6c4424e8b725235` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:56.608366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783bc12-82fd-4fb6-98e6-8c3920398996 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a930a76ad7764359b6c4424e8b725235)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:56.610176 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783bc12-82fd-4fb6-98e6-8c3920398996 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:56.845720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-0783bc12-82fd-4fb6-98e6-8c3920398996 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a930a76ad7764359b6c4424e8b725235'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:56.992145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-0783bc12-82fd-4fb6-98e6-8c3920398996 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a930a76ad7764359b6c4424e8b725235'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:56.992771 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-0783bc12-82fd-4fb6-98e6-8c3920398996 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a930a76ad7764359b6c4424e8b725235'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:56.994146 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-0783bc12-82fd-4fb6-98e6-8c3920398996 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a930a76ad7764359b6c4424e8b725235'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:57.021308 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-0783bc12-82fd-4fb6-98e6-8c3920398996 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a930a76ad7764359b6c4424e8b725235'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:57.044166 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d596b55-bf2c-4083-9b69-c161b53b96ef admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:57.045941 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-1d596b55-bf2c-4083-9b69-c161b53b96ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:57.051668 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 998/2018] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:04:56 2022] DELETE /identity/v3/users/a930a76ad7764359b6c4424e8b725235 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:57.176704 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d596b55-bf2c-4083-9b69-c161b53b96ef admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:57.248640 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4cb9d-9a4c-4f54-85d9-63a9ea1248b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:57.249378 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4cb9d-9a4c-4f54-85d9-63a9ea1248b6 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:57.249661 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4cb9d-9a4c-4f54-85d9-63a9ea1248b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:57.250059 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4cb9d-9a4c-4f54-85d9-63a9ea1248b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:57.258222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-3cc4cb9d-9a4c-4f54-85d9-63a9ea1248b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:57.403669 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4cb9d-9a4c-4f54-85d9-63a9ea1248b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:57.407164 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4cb9d-9a4c-4f54-85d9-63a9ea1248b6 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:57.418719 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1021/2019] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:04:57 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:04:57.983644 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783bc12-82fd-4fb6-98e6-8c3920398996 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:57.985771 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-0783bc12-82fd-4fb6-98e6-8c3920398996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:58.078822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783bc12-82fd-4fb6-98e6-8c3920398996 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:58.086990 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c98167-defe-44fa-a29f-19e353b86b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:58.087814 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c98167-defe-44fa-a29f-19e353b86b4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:58.088121 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c98167-defe-44fa-a29f-19e353b86b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:58.088994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c98167-defe-44fa-a29f-19e353b86b4b admin admin] PATH_INFO: `/v3/projects/056112a0e10049e3b726f28b6709eecd` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:58.107290 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c98167-defe-44fa-a29f-19e353b86b4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=056112a0e10049e3b726f28b6709eecd)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:58.112726 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c98167-defe-44fa-a29f-19e353b86b4b admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:58.245020 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-89c98167-defe-44fa-a29f-19e353b86b4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '056112a0e10049e3b726f28b6709eecd', 'request_id': 'req-89c98167-defe-44fa-a29f-19e353b86b4b'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:58.255771 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-89c98167-defe-44fa-a29f-19e353b86b4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '056112a0e10049e3b726f28b6709eecd', 'request_id': 'req-89c98167-defe-44fa-a29f-19e353b86b4b'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:58.264303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-89c98167-defe-44fa-a29f-19e353b86b4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '056112a0e10049e3b726f28b6709eecd', 'request_id': 'req-89c98167-defe-44fa-a29f-19e353b86b4b'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:58.269135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:58.370641 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-89c98167-defe-44fa-a29f-19e353b86b4b admin admin] The token cache is being invalidate because project 056112a0e10049e3b726f28b6709eecd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:04:58.370799 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-89c98167-defe-44fa-a29f-19e353b86b4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:58.384474 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-89c98167-defe-44fa-a29f-19e353b86b4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:04:58.395308 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 999/2020] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:04:57 2022] DELETE /identity/v3/projects/056112a0e10049e3b726f28b6709eecd => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:58.397109 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59119f0-ec4d-41f8-b94c-4aa89725e15e None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:58.397554 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59119f0-ec4d-41f8-b94c-4aa89725e15e None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:58.397649 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59119f0-ec4d-41f8-b94c-4aa89725e15e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:58.420656 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-b59119f0-ec4d-41f8-b94c-4aa89725e15e None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:58.421936 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-b59119f0-ec4d-41f8-b94c-4aa89725e15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:58.505333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1022/2021] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:58.513388 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66444919-1b08-4db1-9a02-b86cb575a489 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:58.513672 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66444919-1b08-4db1-9a02-b86cb575a489 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:58.513995 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66444919-1b08-4db1-9a02-b86cb575a489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:58.535904 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-66444919-1b08-4db1-9a02-b86cb575a489 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:58.544602 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-66444919-1b08-4db1-9a02-b86cb575a489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:58.548035 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1000/2022] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:58.552495 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59119f0-ec4d-41f8-b94c-4aa89725e15e None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:58.563856 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59119f0-ec4d-41f8-b94c-4aa89725e15e None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:58.568502 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97be439-238f-4c15-b920-d5c4656c9865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:58.569147 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97be439-238f-4c15-b920-d5c4656c9865 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:58.569424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97be439-238f-4c15-b920-d5c4656c9865 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:58.569688 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97be439-238f-4c15-b920-d5c4656c9865 admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:58.570359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97be439-238f-4c15-b920-d5c4656c9865 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:04:58.570697 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97be439-238f-4c15-b920-d5c4656c9865 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:58.572706 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97be439-238f-4c15-b920-d5c4656c9865 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:58.574468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:58.585525 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1023/2023] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:04:58.589957 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66444919-1b08-4db1-9a02-b86cb575a489 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:58.599654 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66444919-1b08-4db1-9a02-b86cb575a489 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:58.606225 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b40dd3-417a-4a30-baff-084a7fbe7e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:58.606873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b40dd3-417a-4a30-baff-084a7fbe7e5e admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:58.607146 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b40dd3-417a-4a30-baff-084a7fbe7e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:58.607411 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b40dd3-417a-4a30-baff-084a7fbe7e5e admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:58.608057 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b40dd3-417a-4a30-baff-084a7fbe7e5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:58.609940 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b40dd3-417a-4a30-baff-084a7fbe7e5e admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:58.742825 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1001/2024] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:04:58 2022] POST /identity/v3/projects => generated 383 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:04:58.744302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97be439-238f-4c15-b920-d5c4656c9865 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:58.754584 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97be439-238f-4c15-b920-d5c4656c9865 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:58.816687 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63767f6-7f62-4a72-8c86-2ec9800e9b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:58.817307 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63767f6-7f62-4a72-8c86-2ec9800e9b56 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:58.817884 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63767f6-7f62-4a72-8c86-2ec9800e9b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:58.817884 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63767f6-7f62-4a72-8c86-2ec9800e9b56 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:58.818570 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63767f6-7f62-4a72-8c86-2ec9800e9b56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:58.820675 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63767f6-7f62-4a72-8c86-2ec9800e9b56 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:58.928278 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1024/2025] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:04:58 2022] POST /identity/v3/users => generated 292 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:04:58.932513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b40dd3-417a-4a30-baff-084a7fbe7e5e admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:58.943740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b40dd3-417a-4a30-baff-084a7fbe7e5e admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:58.948747 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc24d1-23eb-4562-b17c-4f728e0c363a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:58.949362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc24d1-23eb-4562-b17c-4f728e0c363a admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:58.949645 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc24d1-23eb-4562-b17c-4f728e0c363a admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:58.949913 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc24d1-23eb-4562-b17c-4f728e0c363a admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:58.950612 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc24d1-23eb-4562-b17c-4f728e0c363a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:58.952255 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc24d1-23eb-4562-b17c-4f728e0c363a admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:58.966173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1002/2026] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:04:58.967452 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63767f6-7f62-4a72-8c86-2ec9800e9b56 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:58.978974 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63767f6-7f62-4a72-8c86-2ec9800e9b56 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:58.984468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645a0c9-a6f0-4ba4-be65-34b6c5eb61da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:58.985071 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645a0c9-a6f0-4ba4-be65-34b6c5eb61da admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:58.985364 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645a0c9-a6f0-4ba4-be65-34b6c5eb61da admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:58.985622 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645a0c9-a6f0-4ba4-be65-34b6c5eb61da admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:58.986264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4645a0c9-a6f0-4ba4-be65-34b6c5eb61da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:58.988013 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4645a0c9-a6f0-4ba4-be65-34b6c5eb61da admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:59.002033 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1025/2027] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:04:59.003458 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc24d1-23eb-4562-b17c-4f728e0c363a admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:59.013711 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc24d1-23eb-4562-b17c-4f728e0c363a admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:59.018433 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b39054-6b21-40aa-9f6a-9107265e99f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:59.019087 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b39054-6b21-40aa-9f6a-9107265e99f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:59.019477 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b39054-6b21-40aa-9f6a-9107265e99f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:59.019632 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b39054-6b21-40aa-9f6a-9107265e99f7 admin admin] PATH_INFO: `/v3/projects/77f1136a42a04fd5a0bc34747e1b0077/users/97dc2d1d4a6f4dfc84a67058ff1a9fc7/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:59.037081 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b39054-6b21-40aa-9f6a-9107265e99f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77f1136a42a04fd5a0bc34747e1b0077, user_id=97dc2d1d4a6f4dfc84a67058ff1a9fc7, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:59.039968 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b39054-6b21-40aa-9f6a-9107265e99f7 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:59.213051 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1003/2028] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:04:59 2022] PUT /identity/v3/projects/77f1136a42a04fd5a0bc34747e1b0077/users/97dc2d1d4a6f4dfc84a67058ff1a9fc7/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:04:59.225513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b184e2-3d60-4dd5-91d1-b8672ece49b9 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:59.225828 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b184e2-3d60-4dd5-91d1-b8672ece49b9 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:59.226096 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b184e2-3d60-4dd5-91d1-b8672ece49b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:59.259302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-09b184e2-3d60-4dd5-91d1-b8672ece49b9 None None] MFA Rules not processed for user `97dc2d1d4a6f4dfc84a67058ff1a9fc7`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:04:59.260973 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-09b184e2-3d60-4dd5-91d1-b8672ece49b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:04:59.333546 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1026/2029] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:04:59.337957 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b39054-6b21-40aa-9f6a-9107265e99f7 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:59.408021 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b39054-6b21-40aa-9f6a-9107265e99f7 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:59.413053 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8a684-b8a1-48f5-96d2-15b123344519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:59.413798 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8a684-b8a1-48f5-96d2-15b123344519 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:59.413935 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8a684-b8a1-48f5-96d2-15b123344519 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:59.414206 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8a684-b8a1-48f5-96d2-15b123344519 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:59.414968 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8a684-b8a1-48f5-96d2-15b123344519 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:59.416912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8a684-b8a1-48f5-96d2-15b123344519 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:59.530054 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1004/2030] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:04:59 2022] POST /identity/v3/projects => generated 383 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:04:59.534274 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b184e2-3d60-4dd5-91d1-b8672ece49b9 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:59.549620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b184e2-3d60-4dd5-91d1-b8672ece49b9 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:59.603919 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df5ef1-1173-4602-8565-261ae2f5bcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:59.604574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df5ef1-1173-4602-8565-261ae2f5bcce admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:59.604851 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df5ef1-1173-4602-8565-261ae2f5bcce admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:59.605136 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df5ef1-1173-4602-8565-261ae2f5bcce admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:59.605895 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df5ef1-1173-4602-8565-261ae2f5bcce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:59.607854 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df5ef1-1173-4602-8565-261ae2f5bcce admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:59.617035 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:04:59.685831 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1027/2031] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:04:59 2022] POST /identity/v3/users => generated 298 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:04:59.687088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8a684-b8a1-48f5-96d2-15b123344519 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:59.697778 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8a684-b8a1-48f5-96d2-15b123344519 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:59.703650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17340a85-c130-4635-9a6e-eb10f5905317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:59.704256 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17340a85-c130-4635-9a6e-eb10f5905317 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:59.704532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17340a85-c130-4635-9a6e-eb10f5905317 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:59.704800 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17340a85-c130-4635-9a6e-eb10f5905317 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:59.705461 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17340a85-c130-4635-9a6e-eb10f5905317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:59.707071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17340a85-c130-4635-9a6e-eb10f5905317 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:59.718591 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1005/2032] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:04:59.723422 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df5ef1-1173-4602-8565-261ae2f5bcce admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:59.733970 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df5ef1-1173-4602-8565-261ae2f5bcce admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:59.738982 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d469e1fe-543b-4e8f-bd34-d94858d48f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:59.739631 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469e1fe-543b-4e8f-bd34-d94858d48f4e admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:59.739906 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469e1fe-543b-4e8f-bd34-d94858d48f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:59.740177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469e1fe-543b-4e8f-bd34-d94858d48f4e admin admin] PATH_INFO: `/v3/projects/773ac9ced4ca4157a5adc35b5c82b14a/users/3dc4dcd10db6465b97a5f936a4b2a834/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:59.756966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d469e1fe-543b-4e8f-bd34-d94858d48f4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=773ac9ced4ca4157a5adc35b5c82b14a, user_id=3dc4dcd10db6465b97a5f936a4b2a834, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:59.759667 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d469e1fe-543b-4e8f-bd34-d94858d48f4e admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:59.834593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1028/2033] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:04:59 2022] PUT /identity/v3/projects/773ac9ced4ca4157a5adc35b5c82b14a/users/3dc4dcd10db6465b97a5f936a4b2a834/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:59.838637 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17340a85-c130-4635-9a6e-eb10f5905317 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:59.849358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17340a85-c130-4635-9a6e-eb10f5905317 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:59.922139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf35a3-0135-44ef-8fd9-8f075eb714ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:59.922945 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf35a3-0135-44ef-8fd9-8f075eb714ba admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:59.923219 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf35a3-0135-44ef-8fd9-8f075eb714ba admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:59.923485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf35a3-0135-44ef-8fd9-8f075eb714ba admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:59.924127 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf35a3-0135-44ef-8fd9-8f075eb714ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:59.925705 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf35a3-0135-44ef-8fd9-8f075eb714ba admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:59.937492 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1006/2034] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:04:59.941585 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d469e1fe-543b-4e8f-bd34-d94858d48f4e admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:04:59.955621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d469e1fe-543b-4e8f-bd34-d94858d48f4e admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:04:59.960552 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482fc4b6-46ff-4bb7-ad8c-e4fd7e257a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:04:59.961207 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482fc4b6-46ff-4bb7-ad8c-e4fd7e257a9d admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:59.961487 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482fc4b6-46ff-4bb7-ad8c-e4fd7e257a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:59.961756 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482fc4b6-46ff-4bb7-ad8c-e4fd7e257a9d admin admin] PATH_INFO: `/v3/projects/773ac9ced4ca4157a5adc35b5c82b14a/users/3dc4dcd10db6465b97a5f936a4b2a834/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:59.963741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482fc4b6-46ff-4bb7-ad8c-e4fd7e257a9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=773ac9ced4ca4157a5adc35b5c82b14a, user_id=3dc4dcd10db6465b97a5f936a4b2a834, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:04:59.967867 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482fc4b6-46ff-4bb7-ad8c-e4fd7e257a9d admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:04:59.979047 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1029/2035] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:04:59 2022] PUT /identity/v3/projects/773ac9ced4ca4157a5adc35b5c82b14a/users/3dc4dcd10db6465b97a5f936a4b2a834/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:04:59.989850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef298fe-34ba-455e-89ad-7759ebf0e710 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:04:59.990142 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef298fe-34ba-455e-89ad-7759ebf0e710 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:04:59.990439 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef298fe-34ba-455e-89ad-7759ebf0e710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:04:59.998304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:00.021478 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-8ef298fe-34ba-455e-89ad-7759ebf0e710 None None] MFA Rules not processed for user `3dc4dcd10db6465b97a5f936a4b2a834`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:00.022727 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-8ef298fe-34ba-455e-89ad-7759ebf0e710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:00.105606 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1007/2036] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:04:59 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:00.117284 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482fc4b6-46ff-4bb7-ad8c-e4fd7e257a9d admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:00.119069 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-482fc4b6-46ff-4bb7-ad8c-e4fd7e257a9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:00.180315 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482fc4b6-46ff-4bb7-ad8c-e4fd7e257a9d admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:00.184602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655696c-f090-40ad-acaf-de4f97077008 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:00.185212 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655696c-f090-40ad-acaf-de4f97077008 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:00.185493 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655696c-f090-40ad-acaf-de4f97077008 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:00.185766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655696c-f090-40ad-acaf-de4f97077008 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:00.195699 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655696c-f090-40ad-acaf-de4f97077008 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:00.197271 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655696c-f090-40ad-acaf-de4f97077008 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:00.208062 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1030/2037] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:05:00.343519 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef298fe-34ba-455e-89ad-7759ebf0e710 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:00.353415 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef298fe-34ba-455e-89ad-7759ebf0e710 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:00.357385 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfec9d-83be-45f8-8415-9257ec6ff10b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:00.357993 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfec9d-83be-45f8-8415-9257ec6ff10b service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:00.358268 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfec9d-83be-45f8-8415-9257ec6ff10b service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:00.358694 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfec9d-83be-45f8-8415-9257ec6ff10b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:00.431158 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbfec9d-83be-45f8-8415-9257ec6ff10b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:00.433719 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbfec9d-83be-45f8-8415-9257ec6ff10b service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:00.444890 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1008/2038] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:01.053496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655696c-f090-40ad-acaf-de4f97077008 service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:01.127071 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655696c-f090-40ad-acaf-de4f97077008 service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:01.131726 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bdbafc-8954-489d-93b4-dda02e4c4f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:01.132412 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bdbafc-8954-489d-93b4-dda02e4c4f4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:01.132690 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bdbafc-8954-489d-93b4-dda02e4c4f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:01.134126 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bdbafc-8954-489d-93b4-dda02e4c4f4f admin admin] PATH_INFO: `/v3/users/97dc2d1d4a6f4dfc84a67058ff1a9fc7` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:01.134126 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bdbafc-8954-489d-93b4-dda02e4c4f4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=97dc2d1d4a6f4dfc84a67058ff1a9fc7)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:01.136817 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bdbafc-8954-489d-93b4-dda02e4c4f4f admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:01.347463 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-80bdbafc-8954-489d-93b4-dda02e4c4f4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97dc2d1d4a6f4dfc84a67058ff1a9fc7'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:01.357702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:INSERT=4,keystone:SELECT=88,keystone:DELETE=8 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:01.363782 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-80bdbafc-8954-489d-93b4-dda02e4c4f4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97dc2d1d4a6f4dfc84a67058ff1a9fc7'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:01.372418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-80bdbafc-8954-489d-93b4-dda02e4c4f4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97dc2d1d4a6f4dfc84a67058ff1a9fc7'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:01.442192 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-80bdbafc-8954-489d-93b4-dda02e4c4f4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97dc2d1d4a6f4dfc84a67058ff1a9fc7'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:01.442996 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-80bdbafc-8954-489d-93b4-dda02e4c4f4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97dc2d1d4a6f4dfc84a67058ff1a9fc7'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:01.449869 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1031/2039] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:05:01 2022] DELETE /identity/v3/users/97dc2d1d4a6f4dfc84a67058ff1a9fc7 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:01.454444 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ffe46-aa60-49ab-b833-6c292fdbe84f None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:01.454908 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ffe46-aa60-49ab-b833-6c292fdbe84f None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:01.455057 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ffe46-aa60-49ab-b833-6c292fdbe84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:01.474775 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-fa6ffe46-aa60-49ab-b833-6c292fdbe84f None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:01.475991 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-fa6ffe46-aa60-49ab-b833-6c292fdbe84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:01.539809 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1009/2040] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:05:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:05:01.546565 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bdbafc-8954-489d-93b4-dda02e4c4f4f admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:01.548040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-80bdbafc-8954-489d-93b4-dda02e4c4f4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:01.616935 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bdbafc-8954-489d-93b4-dda02e4c4f4f admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:01.621209 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274913ff-1dc2-44d9-aeb1-321fbb41fb78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:01.621800 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274913ff-1dc2-44d9-aeb1-321fbb41fb78 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:01.622078 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274913ff-1dc2-44d9-aeb1-321fbb41fb78 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:01.622346 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274913ff-1dc2-44d9-aeb1-321fbb41fb78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:01.632367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274913ff-1dc2-44d9-aeb1-321fbb41fb78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:01.633929 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274913ff-1dc2-44d9-aeb1-321fbb41fb78 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:01.645248 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1032/2041] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:02.170569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ffe46-aa60-49ab-b833-6c292fdbe84f None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:02.172511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-fa6ffe46-aa60-49ab-b833-6c292fdbe84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:02.189696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ffe46-aa60-49ab-b833-6c292fdbe84f None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:02.194835 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb94c1-371a-4bd7-b95c-10e1b1e32e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:02.201101 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb94c1-371a-4bd7-b95c-10e1b1e32e7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:02.201101 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb94c1-371a-4bd7-b95c-10e1b1e32e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:02.201101 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb94c1-371a-4bd7-b95c-10e1b1e32e7b admin admin] PATH_INFO: `/v3/projects/77f1136a42a04fd5a0bc34747e1b0077` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:02.201101 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdb94c1-371a-4bd7-b95c-10e1b1e32e7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=77f1136a42a04fd5a0bc34747e1b0077)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:02.201101 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdb94c1-371a-4bd7-b95c-10e1b1e32e7b admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:02.222184 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:02.320514 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-cfdb94c1-371a-4bd7-b95c-10e1b1e32e7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77f1136a42a04fd5a0bc34747e1b0077', 'request_id': 'req-cfdb94c1-371a-4bd7-b95c-10e1b1e32e7b'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:02.327663 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-cfdb94c1-371a-4bd7-b95c-10e1b1e32e7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77f1136a42a04fd5a0bc34747e1b0077', 'request_id': 'req-cfdb94c1-371a-4bd7-b95c-10e1b1e32e7b'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:02.335801 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-cfdb94c1-371a-4bd7-b95c-10e1b1e32e7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77f1136a42a04fd5a0bc34747e1b0077', 'request_id': 'req-cfdb94c1-371a-4bd7-b95c-10e1b1e32e7b'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:02.396619 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-cfdb94c1-371a-4bd7-b95c-10e1b1e32e7b admin admin] The token cache is being invalidate because project 77f1136a42a04fd5a0bc34747e1b0077 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:05:02.396979 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-cfdb94c1-371a-4bd7-b95c-10e1b1e32e7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:02.397767 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-cfdb94c1-371a-4bd7-b95c-10e1b1e32e7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:02.399580 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1010/2042] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:05:02 2022] DELETE /identity/v3/projects/77f1136a42a04fd5a0bc34747e1b0077 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:02.404414 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274913ff-1dc2-44d9-aeb1-321fbb41fb78 service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:02.414090 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274913ff-1dc2-44d9-aeb1-321fbb41fb78 service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:02.485420 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a3829-cf14-4ad3-8980-0af0f2424ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:02.486062 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a3829-cf14-4ad3-8980-0af0f2424ee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:02.486339 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a3829-cf14-4ad3-8980-0af0f2424ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:02.486638 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a3829-cf14-4ad3-8980-0af0f2424ee7 admin admin] PATH_INFO: `/v3/users/3dc4dcd10db6465b97a5f936a4b2a834` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:02.487799 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a3829-cf14-4ad3-8980-0af0f2424ee7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dc4dcd10db6465b97a5f936a4b2a834)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:02.489698 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a3829-cf14-4ad3-8980-0af0f2424ee7 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:02.686420 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ce1a3829-cf14-4ad3-8980-0af0f2424ee7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dc4dcd10db6465b97a5f936a4b2a834'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:02.699723 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ce1a3829-cf14-4ad3-8980-0af0f2424ee7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dc4dcd10db6465b97a5f936a4b2a834'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:02.708025 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ce1a3829-cf14-4ad3-8980-0af0f2424ee7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dc4dcd10db6465b97a5f936a4b2a834'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:02.790628 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ce1a3829-cf14-4ad3-8980-0af0f2424ee7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dc4dcd10db6465b97a5f936a4b2a834'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:02.791258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ce1a3829-cf14-4ad3-8980-0af0f2424ee7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dc4dcd10db6465b97a5f936a4b2a834'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:02.793903 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:02.799537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1033/2043] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:05:02 2022] DELETE /identity/v3/users/3dc4dcd10db6465b97a5f936a4b2a834 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:03.262615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb94c1-371a-4bd7-b95c-10e1b1e32e7b admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:03.263358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-cfdb94c1-371a-4bd7-b95c-10e1b1e32e7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:03.282463 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb94c1-371a-4bd7-b95c-10e1b1e32e7b admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:03.344014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac32c149-1174-4641-bbda-6c313f9b99b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:03.344662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac32c149-1174-4641-bbda-6c313f9b99b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:03.344938 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac32c149-1174-4641-bbda-6c313f9b99b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:03.345206 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac32c149-1174-4641-bbda-6c313f9b99b2 admin admin] PATH_INFO: `/v3/projects/773ac9ced4ca4157a5adc35b5c82b14a` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:03.346357 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac32c149-1174-4641-bbda-6c313f9b99b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=773ac9ced4ca4157a5adc35b5c82b14a)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:03.348293 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac32c149-1174-4641-bbda-6c313f9b99b2 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:03.486527 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-ac32c149-1174-4641-bbda-6c313f9b99b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '773ac9ced4ca4157a5adc35b5c82b14a', 'request_id': 'req-ac32c149-1174-4641-bbda-6c313f9b99b2'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:03.495787 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-ac32c149-1174-4641-bbda-6c313f9b99b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '773ac9ced4ca4157a5adc35b5c82b14a', 'request_id': 'req-ac32c149-1174-4641-bbda-6c313f9b99b2'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:03.507169 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-ac32c149-1174-4641-bbda-6c313f9b99b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '773ac9ced4ca4157a5adc35b5c82b14a', 'request_id': 'req-ac32c149-1174-4641-bbda-6c313f9b99b2'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:03.583079 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-ac32c149-1174-4641-bbda-6c313f9b99b2 admin admin] The token cache is being invalidate because project 773ac9ced4ca4157a5adc35b5c82b14a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:05:03.583355 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-ac32c149-1174-4641-bbda-6c313f9b99b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:03.584022 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-ac32c149-1174-4641-bbda-6c313f9b99b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:03.585616 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1011/2044] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:05:03 2022] DELETE /identity/v3/projects/773ac9ced4ca4157a5adc35b5c82b14a => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:03.591445 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14c61d-dc20-49ee-a72c-fa58d5f7f915 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:03.591964 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14c61d-dc20-49ee-a72c-fa58d5f7f915 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:03.592147 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14c61d-dc20-49ee-a72c-fa58d5f7f915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:03.616714 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-9c14c61d-dc20-49ee-a72c-fa58d5f7f915 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:03.618676 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-9c14c61d-dc20-49ee-a72c-fa58d5f7f915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:03.701537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1034/2045] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:03.709574 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f1eaa-1db6-4642-990a-a2b8699fde34 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:03.709893 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f1eaa-1db6-4642-990a-a2b8699fde34 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:03.710120 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f1eaa-1db6-4642-990a-a2b8699fde34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:03.736957 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-1f8f1eaa-1db6-4642-990a-a2b8699fde34 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:03.738285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-1f8f1eaa-1db6-4642-990a-a2b8699fde34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:03.747534 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1012/2046] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:03.752167 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14c61d-dc20-49ee-a72c-fa58d5f7f915 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:03.763359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14c61d-dc20-49ee-a72c-fa58d5f7f915 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:03.768746 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3507f2-a172-4d8b-9e66-d3a0102f9981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:03.771204 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3507f2-a172-4d8b-9e66-d3a0102f9981 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:03.771204 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3507f2-a172-4d8b-9e66-d3a0102f9981 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:03.771204 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3507f2-a172-4d8b-9e66-d3a0102f9981 admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:03.771204 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3507f2-a172-4d8b-9e66-d3a0102f9981 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:05:03.771204 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3507f2-a172-4d8b-9e66-d3a0102f9981 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:03.772483 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3507f2-a172-4d8b-9e66-d3a0102f9981 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:03.786341 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1035/2047] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:05:03.791545 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f1eaa-1db6-4642-990a-a2b8699fde34 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:03.802403 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f1eaa-1db6-4642-990a-a2b8699fde34 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:03.807288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b97f28-a2e8-4ee9-b722-e116a82eaf9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:03.808173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b97f28-a2e8-4ee9-b722-e116a82eaf9f admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:03.808454 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b97f28-a2e8-4ee9-b722-e116a82eaf9f admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:03.808723 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b97f28-a2e8-4ee9-b722-e116a82eaf9f admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:03.809382 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b97f28-a2e8-4ee9-b722-e116a82eaf9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:03.811414 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b97f28-a2e8-4ee9-b722-e116a82eaf9f admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:03.936058 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1013/2048] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:03 2022] POST /identity/v3/projects => generated 371 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:03.939767 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3507f2-a172-4d8b-9e66-d3a0102f9981 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:03.951277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3507f2-a172-4d8b-9e66-d3a0102f9981 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:04.014830 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b3cbe-fd5c-4927-a7d2-2813fd9c0a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:04.015691 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b3cbe-fd5c-4927-a7d2-2813fd9c0a55 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:04.015968 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b3cbe-fd5c-4927-a7d2-2813fd9c0a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:04.016235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b3cbe-fd5c-4927-a7d2-2813fd9c0a55 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:04.016882 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073b3cbe-fd5c-4927-a7d2-2813fd9c0a55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:04.018906 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073b3cbe-fd5c-4927-a7d2-2813fd9c0a55 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:04.127151 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1036/2049] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:03 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:04.132746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b97f28-a2e8-4ee9-b722-e116a82eaf9f admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:04.143697 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b97f28-a2e8-4ee9-b722-e116a82eaf9f admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:04.148294 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74c823-ba71-45da-8417-6bf55c84d8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:04.148949 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74c823-ba71-45da-8417-6bf55c84d8cd admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:04.149510 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74c823-ba71-45da-8417-6bf55c84d8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:04.149776 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74c823-ba71-45da-8417-6bf55c84d8cd admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:04.150445 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e74c823-ba71-45da-8417-6bf55c84d8cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:04.152957 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e74c823-ba71-45da-8417-6bf55c84d8cd admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:04.159266 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:04.164667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1014/2050] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:04.170337 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b3cbe-fd5c-4927-a7d2-2813fd9c0a55 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:04.180404 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b3cbe-fd5c-4927-a7d2-2813fd9c0a55 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:04.187273 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d6b47-11da-4e37-ac5c-e1c7cbd86afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:04.187892 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d6b47-11da-4e37-ac5c-e1c7cbd86afa admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:04.188172 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d6b47-11da-4e37-ac5c-e1c7cbd86afa admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:04.188457 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d6b47-11da-4e37-ac5c-e1c7cbd86afa admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:04.189244 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d6b47-11da-4e37-ac5c-e1c7cbd86afa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:04.190918 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d6b47-11da-4e37-ac5c-e1c7cbd86afa admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:04.202897 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1037/2051] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:04.207842 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74c823-ba71-45da-8417-6bf55c84d8cd admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:04.218861 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74c823-ba71-45da-8417-6bf55c84d8cd admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:04.224173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18bc02-2914-402b-81a7-82d2e2db526c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:04.224914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18bc02-2914-402b-81a7-82d2e2db526c admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:04.225769 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18bc02-2914-402b-81a7-82d2e2db526c admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:04.227177 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18bc02-2914-402b-81a7-82d2e2db526c admin admin] PATH_INFO: `/v3/projects/e13948d5ffa44bfc821896481ba0a8ee/users/72ef88a472e3499e8913a9d29bdcd083/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:04.248216 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db18bc02-2914-402b-81a7-82d2e2db526c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e13948d5ffa44bfc821896481ba0a8ee, user_id=72ef88a472e3499e8913a9d29bdcd083, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:04.250960 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db18bc02-2914-402b-81a7-82d2e2db526c admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:04.322337 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1015/2052] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:04 2022] PUT /identity/v3/projects/e13948d5ffa44bfc821896481ba0a8ee/users/72ef88a472e3499e8913a9d29bdcd083/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:04.334383 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb694e-504d-4c32-9088-a292c6bbfc83 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:04.334701 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb694e-504d-4c32-9088-a292c6bbfc83 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:04.334975 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb694e-504d-4c32-9088-a292c6bbfc83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:04.365864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-5bfb694e-504d-4c32-9088-a292c6bbfc83 None None] MFA Rules not processed for user `72ef88a472e3499e8913a9d29bdcd083`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:04.367152 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-5bfb694e-504d-4c32-9088-a292c6bbfc83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:04.425698 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:04.435874 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1038/2053] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:05:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:04.447819 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18bc02-2914-402b-81a7-82d2e2db526c admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:04.450751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-db18bc02-2914-402b-81a7-82d2e2db526c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:04.509743 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18bc02-2914-402b-81a7-82d2e2db526c admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:04.514087 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616002d-ca47-4870-8efd-00b8221fcc9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:04.514647 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616002d-ca47-4870-8efd-00b8221fcc9f service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:04.514925 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616002d-ca47-4870-8efd-00b8221fcc9f service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:04.515193 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616002d-ca47-4870-8efd-00b8221fcc9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:04.523694 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d616002d-ca47-4870-8efd-00b8221fcc9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:04.525051 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d616002d-ca47-4870-8efd-00b8221fcc9f service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:04.535020 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1016/2054] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:04.556466 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb694e-504d-4c32-9088-a292c6bbfc83 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:04.622508 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb694e-504d-4c32-9088-a292c6bbfc83 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:04.627294 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f00577e-2f37-4232-a7f2-68e285d1b30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:04.627932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00577e-2f37-4232-a7f2-68e285d1b30c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:04.628208 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00577e-2f37-4232-a7f2-68e285d1b30c admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:04.628612 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00577e-2f37-4232-a7f2-68e285d1b30c admin admin] PATH_INFO: `/v3/users/72ef88a472e3499e8913a9d29bdcd083` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:04.629752 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f00577e-2f37-4232-a7f2-68e285d1b30c admin admin] RBAC: Authorizing `identity:delete_user(user_id=72ef88a472e3499e8913a9d29bdcd083)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:04.631725 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f00577e-2f37-4232-a7f2-68e285d1b30c admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:04.841577 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616002d-ca47-4870-8efd-00b8221fcc9f service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:04.843043 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-d616002d-ca47-4870-8efd-00b8221fcc9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:04.858173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616002d-ca47-4870-8efd-00b8221fcc9f service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:04.862865 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552b2ff-fcce-48a3-9a15-6a260fe7c200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:04.863507 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552b2ff-fcce-48a3-9a15-6a260fe7c200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:04.863785 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552b2ff-fcce-48a3-9a15-6a260fe7c200 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:04.864057 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552b2ff-fcce-48a3-9a15-6a260fe7c200 admin admin] PATH_INFO: `/v3/users/59bd4cdf60094d2bb134e2852aec7959` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:04.865197 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552b2ff-fcce-48a3-9a15-6a260fe7c200 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59bd4cdf60094d2bb134e2852aec7959)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:04.867165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552b2ff-fcce-48a3-9a15-6a260fe7c200 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:04.898157 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-1f00577e-2f37-4232-a7f2-68e285d1b30c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72ef88a472e3499e8913a9d29bdcd083'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:04.911458 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-1f00577e-2f37-4232-a7f2-68e285d1b30c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72ef88a472e3499e8913a9d29bdcd083'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:04.919245 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-1f00577e-2f37-4232-a7f2-68e285d1b30c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72ef88a472e3499e8913a9d29bdcd083'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:05.024631 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-1f00577e-2f37-4232-a7f2-68e285d1b30c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72ef88a472e3499e8913a9d29bdcd083'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:05.025308 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-1f00577e-2f37-4232-a7f2-68e285d1b30c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72ef88a472e3499e8913a9d29bdcd083'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:05.031877 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1039/2055] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:05:04 2022] DELETE /identity/v3/users/72ef88a472e3499e8913a9d29bdcd083 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:05.036405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58210e-720a-4f1c-a990-0eda474a273a None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:05.036699 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58210e-720a-4f1c-a990-0eda474a273a None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:05.036977 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58210e-720a-4f1c-a990-0eda474a273a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:05.057067 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-bb58210e-720a-4f1c-a990-0eda474a273a None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:05.058750 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-bb58210e-720a-4f1c-a990-0eda474a273a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:05.090512 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-8552b2ff-fcce-48a3-9a15-6a260fe7c200 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59bd4cdf60094d2bb134e2852aec7959'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:05.121007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1040/2056] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:05:05.130978 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58210e-720a-4f1c-a990-0eda474a273a None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:05.132470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-bb58210e-720a-4f1c-a990-0eda474a273a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:05.146698 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:05.184486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-8552b2ff-fcce-48a3-9a15-6a260fe7c200 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59bd4cdf60094d2bb134e2852aec7959'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:05.185108 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-8552b2ff-fcce-48a3-9a15-6a260fe7c200 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59bd4cdf60094d2bb134e2852aec7959'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:05.185724 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-8552b2ff-fcce-48a3-9a15-6a260fe7c200 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59bd4cdf60094d2bb134e2852aec7959'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:05.191207 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58210e-720a-4f1c-a990-0eda474a273a None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:05.194931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5efbb-f2d2-4725-9768-f27897574aec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:05.195540 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5efbb-f2d2-4725-9768-f27897574aec service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:05.195817 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5efbb-f2d2-4725-9768-f27897574aec service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:05.196097 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5efbb-f2d2-4725-9768-f27897574aec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:05.197841 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-74e5efbb-f2d2-4725-9768-f27897574aec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:05.199193 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-8552b2ff-fcce-48a3-9a15-6a260fe7c200 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59bd4cdf60094d2bb134e2852aec7959'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:05.210652 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5efbb-f2d2-4725-9768-f27897574aec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:05.212230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5efbb-f2d2-4725-9768-f27897574aec service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:05.212362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1017/2057] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:05:04 2022] DELETE /identity/v3/users/59bd4cdf60094d2bb134e2852aec7959 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:05:05.217433 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343bdc58-5bda-468e-be06-a530d4d23d88 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:05.217771 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343bdc58-5bda-468e-be06-a530d4d23d88 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:05.218044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343bdc58-5bda-468e-be06-a530d4d23d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:05.224876 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1041/2058] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:05.239442 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-343bdc58-5bda-468e-be06-a530d4d23d88 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:05.240718 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-343bdc58-5bda-468e-be06-a530d4d23d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:05.291090 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:05.310018 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1018/2059] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:05:05.320585 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5efbb-f2d2-4725-9768-f27897574aec service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:05.322699 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-74e5efbb-f2d2-4725-9768-f27897574aec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:05.335614 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5efbb-f2d2-4725-9768-f27897574aec service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:05.395769 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28770442-2867-4f27-a71f-3f48f0b11c7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:05.396392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28770442-2867-4f27-a71f-3f48f0b11c7b service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:05.396671 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28770442-2867-4f27-a71f-3f48f0b11c7b service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:05.396941 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28770442-2867-4f27-a71f-3f48f0b11c7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:05.410664 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28770442-2867-4f27-a71f-3f48f0b11c7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:05.415440 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28770442-2867-4f27-a71f-3f48f0b11c7b service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:05.428772 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1042/2060] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:05.728571 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343bdc58-5bda-468e-be06-a530d4d23d88 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:05.730350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-343bdc58-5bda-468e-be06-a530d4d23d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:05.742876 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343bdc58-5bda-468e-be06-a530d4d23d88 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:05.759556 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1be62-4d72-4020-8a61-60f4fccf62f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:05.760285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1be62-4d72-4020-8a61-60f4fccf62f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:05.763308 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1be62-4d72-4020-8a61-60f4fccf62f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:05.763308 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1be62-4d72-4020-8a61-60f4fccf62f3 admin admin] PATH_INFO: `/v3/projects/e13948d5ffa44bfc821896481ba0a8ee` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:05.763308 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc1be62-4d72-4020-8a61-60f4fccf62f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e13948d5ffa44bfc821896481ba0a8ee)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:05.764533 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc1be62-4d72-4020-8a61-60f4fccf62f3 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:05.901803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-9cc1be62-4d72-4020-8a61-60f4fccf62f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e13948d5ffa44bfc821896481ba0a8ee', 'request_id': 'req-9cc1be62-4d72-4020-8a61-60f4fccf62f3'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:05.909643 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-9cc1be62-4d72-4020-8a61-60f4fccf62f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e13948d5ffa44bfc821896481ba0a8ee', 'request_id': 'req-9cc1be62-4d72-4020-8a61-60f4fccf62f3'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:05.927939 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-9cc1be62-4d72-4020-8a61-60f4fccf62f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e13948d5ffa44bfc821896481ba0a8ee', 'request_id': 'req-9cc1be62-4d72-4020-8a61-60f4fccf62f3'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:05.991850 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28770442-2867-4f27-a71f-3f48f0b11c7b service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:05.994307 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-28770442-2867-4f27-a71f-3f48f0b11c7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:06.016856 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28770442-2867-4f27-a71f-3f48f0b11c7b service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:06.030931 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-9cc1be62-4d72-4020-8a61-60f4fccf62f3 admin admin] The token cache is being invalidate because project e13948d5ffa44bfc821896481ba0a8ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:05:06.031269 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-9cc1be62-4d72-4020-8a61-60f4fccf62f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:06.034012 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-9cc1be62-4d72-4020-8a61-60f4fccf62f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:06.034012 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1019/2061] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:05:05 2022] DELETE /identity/v3/projects/e13948d5ffa44bfc821896481ba0a8ee => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:06.038262 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c882f-6415-4d60-b6b8-c179592d2333 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:06.038579 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c882f-6415-4d60-b6b8-c179592d2333 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:06.038857 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c882f-6415-4d60-b6b8-c179592d2333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:06.059904 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-190c882f-6415-4d60-b6b8-c179592d2333 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:06.061117 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-190c882f-6415-4d60-b6b8-c179592d2333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:06.089129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd10404b-9bab-4163-a901-712bffedace1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:06.089718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10404b-9bab-4163-a901-712bffedace1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:06.089891 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10404b-9bab-4163-a901-712bffedace1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:06.090165 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10404b-9bab-4163-a901-712bffedace1 admin admin] PATH_INFO: `/v3/projects/32f667fd1dbb4f6d8b4756abd70ee99d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:06.091385 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd10404b-9bab-4163-a901-712bffedace1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32f667fd1dbb4f6d8b4756abd70ee99d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:06.093301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd10404b-9bab-4163-a901-712bffedace1 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:06.145451 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1020/2062] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:05:06.153915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74111af-330a-44b4-87e5-d0da9123e29b None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:06.154271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74111af-330a-44b4-87e5-d0da9123e29b None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:06.154592 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74111af-330a-44b4-87e5-d0da9123e29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:06.174156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-a74111af-330a-44b4-87e5-d0da9123e29b None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:06.175425 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a74111af-330a-44b4-87e5-d0da9123e29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:06.184032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1021/2063] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:06.188357 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74111af-330a-44b4-87e5-d0da9123e29b None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:06.199632 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74111af-330a-44b4-87e5-d0da9123e29b None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:06.210833 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cd10404b-9bab-4163-a901-712bffedace1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32f667fd1dbb4f6d8b4756abd70ee99d', 'request_id': 'req-cd10404b-9bab-4163-a901-712bffedace1'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:06.265608 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db54b0-4932-4824-8eef-b9c19c31a7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:06.266267 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db54b0-4932-4824-8eef-b9c19c31a7a1 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:06.266614 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db54b0-4932-4824-8eef-b9c19c31a7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:06.266889 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db54b0-4932-4824-8eef-b9c19c31a7a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:06.267606 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db54b0-4932-4824-8eef-b9c19c31a7a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:05:06.267914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db54b0-4932-4824-8eef-b9c19c31a7a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:06.269568 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db54b0-4932-4824-8eef-b9c19c31a7a1 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:06.281192 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cd10404b-9bab-4163-a901-712bffedace1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32f667fd1dbb4f6d8b4756abd70ee99d', 'request_id': 'req-cd10404b-9bab-4163-a901-712bffedace1'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:06.282018 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1022/2064] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:05:06.286240 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:06.287223 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db54b0-4932-4824-8eef-b9c19c31a7a1 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:06.288344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cd10404b-9bab-4163-a901-712bffedace1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32f667fd1dbb4f6d8b4756abd70ee99d', 'request_id': 'req-cd10404b-9bab-4163-a901-712bffedace1'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:06.292528 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:06.297658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db54b0-4932-4824-8eef-b9c19c31a7a1 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:06.301123 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cd10404b-9bab-4163-a901-712bffedace1 admin admin] The token cache is being invalidate because project 32f667fd1dbb4f6d8b4756abd70ee99d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:05:06.301423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cd10404b-9bab-4163-a901-712bffedace1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:06.302070 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cd10404b-9bab-4163-a901-712bffedace1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:06.302575 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec1378-d310-4924-b35c-7c6ede8f8c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:06.303180 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec1378-d310-4924-b35c-7c6ede8f8c2e admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:06.303460 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec1378-d310-4924-b35c-7c6ede8f8c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:06.303639 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1043/2065] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:05:05 2022] DELETE /identity/v3/projects/32f667fd1dbb4f6d8b4756abd70ee99d => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:05:06.303758 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec1378-d310-4924-b35c-7c6ede8f8c2e admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:06.304357 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec1378-d310-4924-b35c-7c6ede8f8c2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:06.306285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec1378-d310-4924-b35c-7c6ede8f8c2e admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:06.308946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd10404b-9bab-4163-a901-712bffedace1 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:06.310617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-cd10404b-9bab-4163-a901-712bffedace1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:06.324308 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd10404b-9bab-4163-a901-712bffedace1 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:06.329470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf89382-fb24-4df4-af19-a5aee62f2c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:06.330187 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf89382-fb24-4df4-af19-a5aee62f2c72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:06.330491 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf89382-fb24-4df4-af19-a5aee62f2c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:06.330763 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf89382-fb24-4df4-af19-a5aee62f2c72 admin admin] PATH_INFO: `/v3/users/83144d6ae2854c39b198dc75c3a9075b` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:06.331913 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf89382-fb24-4df4-af19-a5aee62f2c72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83144d6ae2854c39b198dc75c3a9075b)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:06.333839 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf89382-fb24-4df4-af19-a5aee62f2c72 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:06.436755 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1023/2066] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:06 2022] POST /identity/v3/projects => generated 375 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:05:06.437929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec1378-d310-4924-b35c-7c6ede8f8c2e admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:06.439845 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-f1ec1378-d310-4924-b35c-7c6ede8f8c2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:06.507262 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec1378-d310-4924-b35c-7c6ede8f8c2e admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:06.512946 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f0ae5-f3d2-4951-8d92-1dfae8c9b6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:06.513562 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f0ae5-f3d2-4951-8d92-1dfae8c9b6d9 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:06.513837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f0ae5-f3d2-4951-8d92-1dfae8c9b6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:06.514104 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f0ae5-f3d2-4951-8d92-1dfae8c9b6d9 admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:06.514764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f0ae5-f3d2-4951-8d92-1dfae8c9b6d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:06.516667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f0ae5-f3d2-4951-8d92-1dfae8c9b6d9 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:06.540538 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cdf89382-fb24-4df4-af19-a5aee62f2c72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83144d6ae2854c39b198dc75c3a9075b'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:06.559102 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cdf89382-fb24-4df4-af19-a5aee62f2c72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83144d6ae2854c39b198dc75c3a9075b'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:06.565982 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cdf89382-fb24-4df4-af19-a5aee62f2c72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83144d6ae2854c39b198dc75c3a9075b'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:06.584696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1024/2067] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:06 2022] POST /identity/v3/users => generated 288 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:06.588882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f0ae5-f3d2-4951-8d92-1dfae8c9b6d9 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:06.598893 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f0ae5-f3d2-4951-8d92-1dfae8c9b6d9 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:06.604566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bcd0d8-b7a4-49af-b532-09f3b5e777e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:06.605434 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bcd0d8-b7a4-49af-b532-09f3b5e777e2 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:06.605849 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bcd0d8-b7a4-49af-b532-09f3b5e777e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:06.606272 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bcd0d8-b7a4-49af-b532-09f3b5e777e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:06.607069 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bcd0d8-b7a4-49af-b532-09f3b5e777e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:06.608874 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bcd0d8-b7a4-49af-b532-09f3b5e777e2 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:06.619392 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1025/2068] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:05:06.624221 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bcd0d8-b7a4-49af-b532-09f3b5e777e2 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:06.634023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bcd0d8-b7a4-49af-b532-09f3b5e777e2 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:06.635933 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cdf89382-fb24-4df4-af19-a5aee62f2c72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83144d6ae2854c39b198dc75c3a9075b'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:06.636589 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cdf89382-fb24-4df4-af19-a5aee62f2c72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83144d6ae2854c39b198dc75c3a9075b'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:06.639361 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d9892-65ae-43de-bfb1-cb498a28b036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:06.640011 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d9892-65ae-43de-bfb1-cb498a28b036 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:06.640291 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d9892-65ae-43de-bfb1-cb498a28b036 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:06.640568 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d9892-65ae-43de-bfb1-cb498a28b036 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:06.641212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d9892-65ae-43de-bfb1-cb498a28b036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:06.642943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d9892-65ae-43de-bfb1-cb498a28b036 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:06.643198 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1044/2069] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:05:06 2022] DELETE /identity/v3/users/83144d6ae2854c39b198dc75c3a9075b => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:05:06.658080 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1026/2070] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:05:06.662407 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf89382-fb24-4df4-af19-a5aee62f2c72 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:06.664082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-cdf89382-fb24-4df4-af19-a5aee62f2c72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:06.738180 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf89382-fb24-4df4-af19-a5aee62f2c72 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:06.745927 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df611dbe-eb85-4e22-a0b6-8b6a524cad94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:06.746647 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df611dbe-eb85-4e22-a0b6-8b6a524cad94 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:06.746781 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df611dbe-eb85-4e22-a0b6-8b6a524cad94 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:06.747089 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df611dbe-eb85-4e22-a0b6-8b6a524cad94 admin admin] PATH_INFO: `/v3/projects/134c3ca2aae74f8e891975cc15d5c5ed/users/db23731be4334003bb57e47b12371b30/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:06.763224 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df611dbe-eb85-4e22-a0b6-8b6a524cad94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=134c3ca2aae74f8e891975cc15d5c5ed, user_id=db23731be4334003bb57e47b12371b30, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:06.766153 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df611dbe-eb85-4e22-a0b6-8b6a524cad94 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:06.877693 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1045/2071] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:06 2022] PUT /identity/v3/projects/134c3ca2aae74f8e891975cc15d5c5ed/users/db23731be4334003bb57e47b12371b30/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:05:06.888904 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95dcc62-30a5-4ba6-90f4-8d9e5040e4c3 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:06.889413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95dcc62-30a5-4ba6-90f4-8d9e5040e4c3 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:06.889413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95dcc62-30a5-4ba6-90f4-8d9e5040e4c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:06.931473 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-d95dcc62-30a5-4ba6-90f4-8d9e5040e4c3 None None] MFA Rules not processed for user `db23731be4334003bb57e47b12371b30`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:06.933494 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-d95dcc62-30a5-4ba6-90f4-8d9e5040e4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:07.009280 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1027/2072] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:05:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:07.021106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df611dbe-eb85-4e22-a0b6-8b6a524cad94 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:07.022963 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-df611dbe-eb85-4e22-a0b6-8b6a524cad94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:07.091942 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:07.112486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df611dbe-eb85-4e22-a0b6-8b6a524cad94 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:07.118368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8778ee-241c-4055-a795-daabe46763cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:07.121391 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8778ee-241c-4055-a795-daabe46763cb service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:07.121391 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8778ee-241c-4055-a795-daabe46763cb service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:07.121674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8778ee-241c-4055-a795-daabe46763cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:07.133790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8778ee-241c-4055-a795-daabe46763cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:07.137268 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8778ee-241c-4055-a795-daabe46763cb service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:07.147319 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1046/2073] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:07.231641 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95dcc62-30a5-4ba6-90f4-8d9e5040e4c3 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:07.233239 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-d95dcc62-30a5-4ba6-90f4-8d9e5040e4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:07.299173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:07.328427 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95dcc62-30a5-4ba6-90f4-8d9e5040e4c3 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:07.333748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5418956d-9cd1-4b90-bd77-272ee3ec1c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:07.334494 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418956d-9cd1-4b90-bd77-272ee3ec1c07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:07.334778 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418956d-9cd1-4b90-bd77-272ee3ec1c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:07.335049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418956d-9cd1-4b90-bd77-272ee3ec1c07 admin admin] PATH_INFO: `/v3/projects/5eef473f80cd4fbbae8935b112129ec1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:07.336176 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5418956d-9cd1-4b90-bd77-272ee3ec1c07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eef473f80cd4fbbae8935b112129ec1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:07.338734 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5418956d-9cd1-4b90-bd77-272ee3ec1c07 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:07.452005 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5418956d-9cd1-4b90-bd77-272ee3ec1c07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eef473f80cd4fbbae8935b112129ec1', 'request_id': 'req-5418956d-9cd1-4b90-bd77-272ee3ec1c07'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:07.459234 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5418956d-9cd1-4b90-bd77-272ee3ec1c07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eef473f80cd4fbbae8935b112129ec1', 'request_id': 'req-5418956d-9cd1-4b90-bd77-272ee3ec1c07'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:07.464976 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5418956d-9cd1-4b90-bd77-272ee3ec1c07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eef473f80cd4fbbae8935b112129ec1', 'request_id': 'req-5418956d-9cd1-4b90-bd77-272ee3ec1c07'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:07.584516 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5418956d-9cd1-4b90-bd77-272ee3ec1c07 admin admin] The token cache is being invalidate because project 5eef473f80cd4fbbae8935b112129ec1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:05:07.584808 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5418956d-9cd1-4b90-bd77-272ee3ec1c07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:07.585450 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5418956d-9cd1-4b90-bd77-272ee3ec1c07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:07.587007 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1028/2074] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:05:07 2022] DELETE /identity/v3/projects/5eef473f80cd4fbbae8935b112129ec1 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:07.593451 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106e6ea-0d48-42d4-9e3e-b57f7e334cbe None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:07.593732 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106e6ea-0d48-42d4-9e3e-b57f7e334cbe None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:07.594002 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106e6ea-0d48-42d4-9e3e-b57f7e334cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:07.615628 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-9106e6ea-0d48-42d4-9e3e-b57f7e334cbe None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:07.617122 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-9106e6ea-0d48-42d4-9e3e-b57f7e334cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:07.676454 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1047/2075] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:07.690928 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90ae23-6da1-4e09-9b71-517e0bbfd047 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:07.690928 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90ae23-6da1-4e09-9b71-517e0bbfd047 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:07.690928 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90ae23-6da1-4e09-9b71-517e0bbfd047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:07.706320 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-ae90ae23-6da1-4e09-9b71-517e0bbfd047 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:07.707554 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-ae90ae23-6da1-4e09-9b71-517e0bbfd047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:07.716691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1029/2076] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:05:07.721264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106e6ea-0d48-42d4-9e3e-b57f7e334cbe None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:07.732702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106e6ea-0d48-42d4-9e3e-b57f7e334cbe None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:07.737280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ff1bd-dd42-4c90-95a5-8e7f609247dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:07.737888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ff1bd-dd42-4c90-95a5-8e7f609247dc admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:07.738168 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ff1bd-dd42-4c90-95a5-8e7f609247dc admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:07.738473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ff1bd-dd42-4c90-95a5-8e7f609247dc admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:07.739134 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ff1bd-dd42-4c90-95a5-8e7f609247dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:05:07.739433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ff1bd-dd42-4c90-95a5-8e7f609247dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:07.741033 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ff1bd-dd42-4c90-95a5-8e7f609247dc admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:07.753787 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1048/2077] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:05:07.758474 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90ae23-6da1-4e09-9b71-517e0bbfd047 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:07.768364 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90ae23-6da1-4e09-9b71-517e0bbfd047 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:07.773114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068aedab-c825-45b5-b475-35ceac7f21b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:07.773754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068aedab-c825-45b5-b475-35ceac7f21b4 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:07.774034 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068aedab-c825-45b5-b475-35ceac7f21b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:07.774304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068aedab-c825-45b5-b475-35ceac7f21b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:07.774981 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068aedab-c825-45b5-b475-35ceac7f21b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:07.777936 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068aedab-c825-45b5-b475-35ceac7f21b4 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:07.923983 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1030/2078] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:07 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:07.926060 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ff1bd-dd42-4c90-95a5-8e7f609247dc admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:07.937530 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ff1bd-dd42-4c90-95a5-8e7f609247dc admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:08.024826 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e02593-a5f6-4447-bf35-5f6bb328a305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:08.025527 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e02593-a5f6-4447-bf35-5f6bb328a305 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:08.028701 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e02593-a5f6-4447-bf35-5f6bb328a305 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:08.028866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e02593-a5f6-4447-bf35-5f6bb328a305 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:08.029740 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e02593-a5f6-4447-bf35-5f6bb328a305 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:08.033798 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e02593-a5f6-4447-bf35-5f6bb328a305 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:08.165201 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1049/2079] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:07 2022] POST /identity/v3/users => generated 289 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:08.166420 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068aedab-c825-45b5-b475-35ceac7f21b4 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:08.176992 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068aedab-c825-45b5-b475-35ceac7f21b4 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:08.181759 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8936d9-a615-4f5e-9ed8-68ed7fc6d96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:08.182466 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8936d9-a615-4f5e-9ed8-68ed7fc6d96f admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:08.182928 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8936d9-a615-4f5e-9ed8-68ed7fc6d96f admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:08.183056 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8936d9-a615-4f5e-9ed8-68ed7fc6d96f admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:08.183668 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8936d9-a615-4f5e-9ed8-68ed7fc6d96f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:08.185288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8936d9-a615-4f5e-9ed8-68ed7fc6d96f admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:08.195704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1031/2080] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:08.200451 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e02593-a5f6-4447-bf35-5f6bb328a305 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:08.210542 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e02593-a5f6-4447-bf35-5f6bb328a305 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:08.215984 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa66ad-577c-4b86-a39f-8fe20aeb7480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:08.216999 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa66ad-577c-4b86-a39f-8fe20aeb7480 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:08.216999 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa66ad-577c-4b86-a39f-8fe20aeb7480 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:08.217166 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa66ad-577c-4b86-a39f-8fe20aeb7480 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:08.217758 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaa66ad-577c-4b86-a39f-8fe20aeb7480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:08.219431 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaa66ad-577c-4b86-a39f-8fe20aeb7480 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:08.236568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1050/2081] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:05:08.240795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8936d9-a615-4f5e-9ed8-68ed7fc6d96f admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:08.250958 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8936d9-a615-4f5e-9ed8-68ed7fc6d96f admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:08.255615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c8b3e-b7d8-4e15-b4bb-5fc5ff6ab124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:08.256297 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c8b3e-b7d8-4e15-b4bb-5fc5ff6ab124 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:08.256584 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c8b3e-b7d8-4e15-b4bb-5fc5ff6ab124 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:08.262124 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c8b3e-b7d8-4e15-b4bb-5fc5ff6ab124 admin admin] PATH_INFO: `/v3/projects/b9f47c1525634bc3b21a807b43552d95/users/8cd414ad803847b7837efc2348c8d848/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:08.273524 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c8b3e-b7d8-4e15-b4bb-5fc5ff6ab124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9f47c1525634bc3b21a807b43552d95, user_id=8cd414ad803847b7837efc2348c8d848, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:08.276903 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c8b3e-b7d8-4e15-b4bb-5fc5ff6ab124 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:08.396414 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1032/2082] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:08 2022] PUT /identity/v3/projects/b9f47c1525634bc3b21a807b43552d95/users/8cd414ad803847b7837efc2348c8d848/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:05:08.412820 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791077f2-6bf5-496f-a340-fe4707f8c0c6 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:08.413125 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791077f2-6bf5-496f-a340-fe4707f8c0c6 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:08.413394 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791077f2-6bf5-496f-a340-fe4707f8c0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:08.444505 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-791077f2-6bf5-496f-a340-fe4707f8c0c6 None None] MFA Rules not processed for user `8cd414ad803847b7837efc2348c8d848`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:08.445750 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-791077f2-6bf5-496f-a340-fe4707f8c0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:08.489357 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:08.513371 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1051/2083] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:05:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:08.525466 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c8b3e-b7d8-4e15-b4bb-5fc5ff6ab124 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:08.526699 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-1a9c8b3e-b7d8-4e15-b4bb-5fc5ff6ab124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:08.584890 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c8b3e-b7d8-4e15-b4bb-5fc5ff6ab124 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:08.589173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc6857d-2b3c-441b-92ea-5c3099edde48 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2e72c8e7b444e978bf99c6404febcd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:08.589917 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6857d-2b3c-441b-92ea-5c3099edde48 service glance] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:08.590204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6857d-2b3c-441b-92ea-5c3099edde48 service glance] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:08.590508 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6857d-2b3c-441b-92ea-5c3099edde48 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:08.599787 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc6857d-2b3c-441b-92ea-5c3099edde48 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:08.601227 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc6857d-2b3c-441b-92ea-5c3099edde48 service glance] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:08.611300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1033/2084] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:17.883224 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791077f2-6bf5-496f-a340-fe4707f8c0c6 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:17.884900 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-791077f2-6bf5-496f-a340-fe4707f8c0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:17.972825 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791077f2-6bf5-496f-a340-fe4707f8c0c6 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:17.978447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c9bf9-a0b8-4aec-b3e4-305e5482f113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:17.979141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c9bf9-a0b8-4aec-b3e4-305e5482f113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:17.979424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c9bf9-a0b8-4aec-b3e4-305e5482f113 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:17.979699 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c9bf9-a0b8-4aec-b3e4-305e5482f113 admin admin] PATH_INFO: `/v3/users/db23731be4334003bb57e47b12371b30` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:17.980875 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69c9bf9-a0b8-4aec-b3e4-305e5482f113 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db23731be4334003bb57e47b12371b30)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:17.982860 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69c9bf9-a0b8-4aec-b3e4-305e5482f113 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:18.269252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c69c9bf9-a0b8-4aec-b3e4-305e5482f113 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db23731be4334003bb57e47b12371b30'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:18.281577 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c69c9bf9-a0b8-4aec-b3e4-305e5482f113 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db23731be4334003bb57e47b12371b30'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:18.289032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c69c9bf9-a0b8-4aec-b3e4-305e5482f113 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db23731be4334003bb57e47b12371b30'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:18.400172 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c69c9bf9-a0b8-4aec-b3e4-305e5482f113 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db23731be4334003bb57e47b12371b30'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:18.400946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c69c9bf9-a0b8-4aec-b3e4-305e5482f113 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db23731be4334003bb57e47b12371b30'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:18.409380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1052/2085] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:05:17 2022] DELETE /identity/v3/users/db23731be4334003bb57e47b12371b30 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:18.414369 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8121ec3-9391-4a4e-a317-e838b00d60f2 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:18.414990 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8121ec3-9391-4a4e-a317-e838b00d60f2 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:18.415084 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8121ec3-9391-4a4e-a317-e838b00d60f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:18.430800 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:18.438562 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-b8121ec3-9391-4a4e-a317-e838b00d60f2 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:18.440150 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-b8121ec3-9391-4a4e-a317-e838b00d60f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:18.505977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1034/2086] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:05:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:18.516629 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c9bf9-a0b8-4aec-b3e4-305e5482f113 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:18.518146 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-c69c9bf9-a0b8-4aec-b3e4-305e5482f113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:18.623946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c9bf9-a0b8-4aec-b3e4-305e5482f113 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:18.628549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4bffd-4a2d-40ea-86d5-b94ceb4d6ecb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:18.629174 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4bffd-4a2d-40ea-86d5-b94ceb4d6ecb service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:18.632101 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4bffd-4a2d-40ea-86d5-b94ceb4d6ecb service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:18.632101 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4bffd-4a2d-40ea-86d5-b94ceb4d6ecb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:18.642365 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d4bffd-4a2d-40ea-86d5-b94ceb4d6ecb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:18.644557 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d4bffd-4a2d-40ea-86d5-b94ceb4d6ecb service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:18.662898 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1053/2087] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:19.188329 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8121ec3-9391-4a4e-a317-e838b00d60f2 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:19.190116 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-b8121ec3-9391-4a4e-a317-e838b00d60f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:19.218721 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8121ec3-9391-4a4e-a317-e838b00d60f2 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:19.224563 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aad78a-ff87-4b85-a9cd-280d6fb06ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:19.225380 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aad78a-ff87-4b85-a9cd-280d6fb06ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:19.225876 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aad78a-ff87-4b85-a9cd-280d6fb06ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:19.226158 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aad78a-ff87-4b85-a9cd-280d6fb06ca0 admin admin] PATH_INFO: `/v3/projects/134c3ca2aae74f8e891975cc15d5c5ed` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:19.227758 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aad78a-ff87-4b85-a9cd-280d6fb06ca0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=134c3ca2aae74f8e891975cc15d5c5ed)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:19.230985 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aad78a-ff87-4b85-a9cd-280d6fb06ca0 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:19.335759 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-e1aad78a-ff87-4b85-a9cd-280d6fb06ca0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '134c3ca2aae74f8e891975cc15d5c5ed', 'request_id': 'req-e1aad78a-ff87-4b85-a9cd-280d6fb06ca0'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:19.343417 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-e1aad78a-ff87-4b85-a9cd-280d6fb06ca0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '134c3ca2aae74f8e891975cc15d5c5ed', 'request_id': 'req-e1aad78a-ff87-4b85-a9cd-280d6fb06ca0'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:19.351976 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-e1aad78a-ff87-4b85-a9cd-280d6fb06ca0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '134c3ca2aae74f8e891975cc15d5c5ed', 'request_id': 'req-e1aad78a-ff87-4b85-a9cd-280d6fb06ca0'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:19.442150 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-e1aad78a-ff87-4b85-a9cd-280d6fb06ca0 admin admin] The token cache is being invalidate because project 134c3ca2aae74f8e891975cc15d5c5ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:05:19.442756 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-e1aad78a-ff87-4b85-a9cd-280d6fb06ca0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:19.443260 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-e1aad78a-ff87-4b85-a9cd-280d6fb06ca0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:19.446450 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1035/2088] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:05:19 2022] DELETE /identity/v3/projects/134c3ca2aae74f8e891975cc15d5c5ed => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:19.450644 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c33a2f-d8db-4262-8190-36b66bf122a9 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:19.451184 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c33a2f-d8db-4262-8190-36b66bf122a9 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:19.451184 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c33a2f-d8db-4262-8190-36b66bf122a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:19.460971 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:19.473979 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-e5c33a2f-d8db-4262-8190-36b66bf122a9 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:19.475289 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-e5c33a2f-d8db-4262-8190-36b66bf122a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:19.565815 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1054/2089] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:19.574469 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10342fd5-0447-43ee-9f53-530c658d19f5 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:19.575141 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10342fd5-0447-43ee-9f53-530c658d19f5 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:19.575740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10342fd5-0447-43ee-9f53-530c658d19f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:19.604923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-10342fd5-0447-43ee-9f53-530c658d19f5 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:19.606305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-10342fd5-0447-43ee-9f53-530c658d19f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:19.616228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1036/2090] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:19.621663 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c33a2f-d8db-4262-8190-36b66bf122a9 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:19.632600 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c33a2f-d8db-4262-8190-36b66bf122a9 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:19.640609 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62252a69-3bd1-4663-975f-cc53a5c236f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:19.641327 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62252a69-3bd1-4663-975f-cc53a5c236f7 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:19.641707 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62252a69-3bd1-4663-975f-cc53a5c236f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:19.641980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62252a69-3bd1-4663-975f-cc53a5c236f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:19.642668 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62252a69-3bd1-4663-975f-cc53a5c236f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:05:19.642964 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62252a69-3bd1-4663-975f-cc53a5c236f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:19.644595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62252a69-3bd1-4663-975f-cc53a5c236f7 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:19.660732 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1055/2091] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:05:19.669407 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10342fd5-0447-43ee-9f53-530c658d19f5 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:19.675817 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10342fd5-0447-43ee-9f53-530c658d19f5 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:19.681448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6139d-6028-4524-8c56-bc56f484e09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:19.682699 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6139d-6028-4524-8c56-bc56f484e09c admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:19.682967 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6139d-6028-4524-8c56-bc56f484e09c admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:19.689783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6139d-6028-4524-8c56-bc56f484e09c admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:19.689783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee6139d-6028-4524-8c56-bc56f484e09c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:19.689783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee6139d-6028-4524-8c56-bc56f484e09c admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:19.795181 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1037/2092] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:19 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:05:19.796510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62252a69-3bd1-4663-975f-cc53a5c236f7 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:19.808335 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62252a69-3bd1-4663-975f-cc53a5c236f7 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:19.875384 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5b3d7-8ef2-4058-a678-a23baec043f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:19.875916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5b3d7-8ef2-4058-a678-a23baec043f2 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:19.876196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5b3d7-8ef2-4058-a678-a23baec043f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:19.876518 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5b3d7-8ef2-4058-a678-a23baec043f2 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:19.877103 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5b3d7-8ef2-4058-a678-a23baec043f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:19.879404 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5b3d7-8ef2-4058-a678-a23baec043f2 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:19.962181 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1056/2093] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:19 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:19.967377 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6139d-6028-4524-8c56-bc56f484e09c admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:19.980040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6139d-6028-4524-8c56-bc56f484e09c admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:19.984835 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3f36e-7d61-4878-96ef-93003221b6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:19.989640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3f36e-7d61-4878-96ef-93003221b6ff admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:19.989640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3f36e-7d61-4878-96ef-93003221b6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:19.989640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3f36e-7d61-4878-96ef-93003221b6ff admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:19.989640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b3f36e-7d61-4878-96ef-93003221b6ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:19.989640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b3f36e-7d61-4878-96ef-93003221b6ff admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:19.998355 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1038/2094] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:20.002333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5b3d7-8ef2-4058-a678-a23baec043f2 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:20.012557 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5b3d7-8ef2-4058-a678-a23baec043f2 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:20.017430 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d5763-f4b9-4a63-b946-dc984be4b9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:20.018325 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d5763-f4b9-4a63-b946-dc984be4b9b9 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:20.018325 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d5763-f4b9-4a63-b946-dc984be4b9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:20.018648 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d5763-f4b9-4a63-b946-dc984be4b9b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:20.019247 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d5763-f4b9-4a63-b946-dc984be4b9b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:20.020841 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d5763-f4b9-4a63-b946-dc984be4b9b9 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:20.032705 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1057/2095] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:20.036964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3f36e-7d61-4878-96ef-93003221b6ff admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:20.048512 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3f36e-7d61-4878-96ef-93003221b6ff admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:20.053318 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761c265-a0ea-4955-b547-9e1b9437e7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:20.053972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761c265-a0ea-4955-b547-9e1b9437e7fe admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:20.054244 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761c265-a0ea-4955-b547-9e1b9437e7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:20.054555 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761c265-a0ea-4955-b547-9e1b9437e7fe admin admin] PATH_INFO: `/v3/projects/fea563d255a74413848ba34c46def500/users/59989a71e17743b9b7d68e269fbb4e88/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:20.069776 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6761c265-a0ea-4955-b547-9e1b9437e7fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=fea563d255a74413848ba34c46def500, user_id=59989a71e17743b9b7d68e269fbb4e88, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:20.072979 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6761c265-a0ea-4955-b547-9e1b9437e7fe admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:20.133728 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1039/2096] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:20 2022] PUT /identity/v3/projects/fea563d255a74413848ba34c46def500/users/59989a71e17743b9b7d68e269fbb4e88/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:20.144334 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839db3c3-e20a-4b6e-b604-ffadc6cff25d None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:20.144629 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839db3c3-e20a-4b6e-b604-ffadc6cff25d None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:20.144903 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839db3c3-e20a-4b6e-b604-ffadc6cff25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:20.186961 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-839db3c3-e20a-4b6e-b604-ffadc6cff25d None None] MFA Rules not processed for user `59989a71e17743b9b7d68e269fbb4e88`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:20.188220 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-839db3c3-e20a-4b6e-b604-ffadc6cff25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:20.253392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:20.257172 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1058/2097] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:05:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:20.262708 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761c265-a0ea-4955-b547-9e1b9437e7fe admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:20.295259 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:20.327694 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761c265-a0ea-4955-b547-9e1b9437e7fe admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:20.332613 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4cbe9-8e13-44c7-8144-5c7bfb1d79ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:20.333305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4cbe9-8e13-44c7-8144-5c7bfb1d79ae admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:20.333588 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4cbe9-8e13-44c7-8144-5c7bfb1d79ae admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:20.333863 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4cbe9-8e13-44c7-8144-5c7bfb1d79ae admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:20.334537 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c4cbe9-8e13-44c7-8144-5c7bfb1d79ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:20.336476 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c4cbe9-8e13-44c7-8144-5c7bfb1d79ae admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:20.460824 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1040/2098] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:20 2022] POST /identity/v3/projects => generated 381 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:05:20.465026 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839db3c3-e20a-4b6e-b604-ffadc6cff25d None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:20.475801 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839db3c3-e20a-4b6e-b604-ffadc6cff25d None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:20.536117 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601b831-0084-4d5d-b420-0496309c0a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:20.536628 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601b831-0084-4d5d-b420-0496309c0a98 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:20.536931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601b831-0084-4d5d-b420-0496309c0a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:20.537183 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601b831-0084-4d5d-b420-0496309c0a98 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:20.537829 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f601b831-0084-4d5d-b420-0496309c0a98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:20.539812 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f601b831-0084-4d5d-b420-0496309c0a98 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:20.645722 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1059/2099] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:20 2022] POST /identity/v3/users => generated 291 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:05:20.647013 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4cbe9-8e13-44c7-8144-5c7bfb1d79ae admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:20.658997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4cbe9-8e13-44c7-8144-5c7bfb1d79ae admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:20.663764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f9d5f-2e90-41a8-a6ee-6309f5da9e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:20.664372 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f9d5f-2e90-41a8-a6ee-6309f5da9e62 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:20.664649 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f9d5f-2e90-41a8-a6ee-6309f5da9e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:20.664926 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f9d5f-2e90-41a8-a6ee-6309f5da9e62 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:20.665735 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f9d5f-2e90-41a8-a6ee-6309f5da9e62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:20.668480 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f9d5f-2e90-41a8-a6ee-6309f5da9e62 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:20.686371 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1041/2100] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:20.687808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601b831-0084-4d5d-b420-0496309c0a98 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:20.702846 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601b831-0084-4d5d-b420-0496309c0a98 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:20.705525 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3d8d9-6be7-468d-a5f7-d0079a0beb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:20.706125 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3d8d9-6be7-468d-a5f7-d0079a0beb9a admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:20.706420 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3d8d9-6be7-468d-a5f7-d0079a0beb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:20.706704 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3d8d9-6be7-468d-a5f7-d0079a0beb9a admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:20.707377 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3d8d9-6be7-468d-a5f7-d0079a0beb9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:20.708970 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3d8d9-6be7-468d-a5f7-d0079a0beb9a admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:20.720344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1060/2101] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:20.724772 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f9d5f-2e90-41a8-a6ee-6309f5da9e62 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:20.735979 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f9d5f-2e90-41a8-a6ee-6309f5da9e62 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:20.741886 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a43c09-3cc5-4bb0-aab3-bf243440413c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:20.742977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a43c09-3cc5-4bb0-aab3-bf243440413c admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:20.743251 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a43c09-3cc5-4bb0-aab3-bf243440413c admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:20.743630 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a43c09-3cc5-4bb0-aab3-bf243440413c admin admin] PATH_INFO: `/v3/projects/5257d9c6b240499eb38d7ea01151d822/users/b01aa659aca14ba3a677f5d4a8944e98/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:20.761221 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a43c09-3cc5-4bb0-aab3-bf243440413c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5257d9c6b240499eb38d7ea01151d822, user_id=b01aa659aca14ba3a677f5d4a8944e98, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:20.763957 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a43c09-3cc5-4bb0-aab3-bf243440413c admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:20.831083 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1042/2102] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:20 2022] PUT /identity/v3/projects/5257d9c6b240499eb38d7ea01151d822/users/b01aa659aca14ba3a677f5d4a8944e98/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:20.843172 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3aced-4e53-42e4-8ce2-0ae2b3b27aea None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:20.843467 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3aced-4e53-42e4-8ce2-0ae2b3b27aea None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:20.843765 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3aced-4e53-42e4-8ce2-0ae2b3b27aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:20.872607 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-2fe3aced-4e53-42e4-8ce2-0ae2b3b27aea None None] MFA Rules not processed for user `b01aa659aca14ba3a677f5d4a8944e98`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:20.873844 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2fe3aced-4e53-42e4-8ce2-0ae2b3b27aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:20.965600 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1061/2103] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:05:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:20.972472 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a43c09-3cc5-4bb0-aab3-bf243440413c admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:21.041638 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a43c09-3cc5-4bb0-aab3-bf243440413c admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:21.046668 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71cace-921c-4977-8c8d-c7910c92fb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:21.047314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71cace-921c-4977-8c8d-c7910c92fb8f admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:21.047594 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71cace-921c-4977-8c8d-c7910c92fb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:21.047867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71cace-921c-4977-8c8d-c7910c92fb8f admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:21.048511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d71cace-921c-4977-8c8d-c7910c92fb8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:21.050896 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d71cace-921c-4977-8c8d-c7910c92fb8f admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:21.159770 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1043/2104] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:20 2022] POST /identity/v3/projects => generated 385 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:05:21.163835 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3aced-4e53-42e4-8ce2-0ae2b3b27aea None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:21.174800 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3aced-4e53-42e4-8ce2-0ae2b3b27aea None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:21.241907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00229657-d319-46f0-aa40-d70dfe680161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:21.241907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00229657-d319-46f0-aa40-d70dfe680161 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:21.241907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00229657-d319-46f0-aa40-d70dfe680161 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:21.241907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00229657-d319-46f0-aa40-d70dfe680161 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:21.242248 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00229657-d319-46f0-aa40-d70dfe680161 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:21.242248 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00229657-d319-46f0-aa40-d70dfe680161 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:21.314680 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:21.321091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1062/2105] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:21 2022] POST /identity/v3/users => generated 299 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:21.325723 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71cace-921c-4977-8c8d-c7910c92fb8f admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:21.336146 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71cace-921c-4977-8c8d-c7910c92fb8f admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:21.341442 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c28d1b9-094e-4e62-a7a6-e34877281d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:21.342046 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28d1b9-094e-4e62-a7a6-e34877281d58 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:21.342375 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28d1b9-094e-4e62-a7a6-e34877281d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:21.343702 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28d1b9-094e-4e62-a7a6-e34877281d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:21.346881 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c28d1b9-094e-4e62-a7a6-e34877281d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:21.346881 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c28d1b9-094e-4e62-a7a6-e34877281d58 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:21.357193 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1044/2106] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:21.361545 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00229657-d319-46f0-aa40-d70dfe680161 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:21.375791 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00229657-d319-46f0-aa40-d70dfe680161 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:21.378027 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27528235-91fa-49f9-b7a9-b72b00a9371c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:21.378723 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27528235-91fa-49f9-b7a9-b72b00a9371c admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:21.379001 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27528235-91fa-49f9-b7a9-b72b00a9371c admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:21.379271 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27528235-91fa-49f9-b7a9-b72b00a9371c admin admin] PATH_INFO: `/v3/projects/f3fc4d9706794cd4bcb7892122251821/users/53e8bd5b3a0f4c8d9e20e6eb0cfef46a/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:21.395978 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27528235-91fa-49f9-b7a9-b72b00a9371c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3fc4d9706794cd4bcb7892122251821, user_id=53e8bd5b3a0f4c8d9e20e6eb0cfef46a, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:21.399480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27528235-91fa-49f9-b7a9-b72b00a9371c admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:21.504809 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1063/2107] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:21 2022] PUT /identity/v3/projects/f3fc4d9706794cd4bcb7892122251821/users/53e8bd5b3a0f4c8d9e20e6eb0cfef46a/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:21.508872 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c28d1b9-094e-4e62-a7a6-e34877281d58 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:21.520313 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c28d1b9-094e-4e62-a7a6-e34877281d58 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:21.569473 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:21.579854 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b9b17-a3cf-45bd-b09e-91da54473ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:21.580459 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b9b17-a3cf-45bd-b09e-91da54473ba3 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:21.580736 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b9b17-a3cf-45bd-b09e-91da54473ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:21.581006 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b9b17-a3cf-45bd-b09e-91da54473ba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:21.581645 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b9b17-a3cf-45bd-b09e-91da54473ba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:21.583295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b9b17-a3cf-45bd-b09e-91da54473ba3 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:21.593416 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1045/2108] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:21.597801 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27528235-91fa-49f9-b7a9-b72b00a9371c admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:21.610024 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27528235-91fa-49f9-b7a9-b72b00a9371c admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:21.615451 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8894564f-336c-435a-9b22-d64e9e2fbef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:21.616124 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894564f-336c-435a-9b22-d64e9e2fbef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:21.617063 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894564f-336c-435a-9b22-d64e9e2fbef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:21.617063 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894564f-336c-435a-9b22-d64e9e2fbef4 admin admin] PATH_INFO: `/v3/projects/f3fc4d9706794cd4bcb7892122251821/users/53e8bd5b3a0f4c8d9e20e6eb0cfef46a/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:21.618751 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8894564f-336c-435a-9b22-d64e9e2fbef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3fc4d9706794cd4bcb7892122251821, user_id=53e8bd5b3a0f4c8d9e20e6eb0cfef46a, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:21.622357 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8894564f-336c-435a-9b22-d64e9e2fbef4 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:21.633434 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1064/2109] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:21 2022] PUT /identity/v3/projects/f3fc4d9706794cd4bcb7892122251821/users/53e8bd5b3a0f4c8d9e20e6eb0cfef46a/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:21.643838 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d88cd5-f13d-4951-92c0-a29a71454088 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:21.644121 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d88cd5-f13d-4951-92c0-a29a71454088 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:21.645956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d88cd5-f13d-4951-92c0-a29a71454088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:21.673680 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-68d88cd5-f13d-4951-92c0-a29a71454088 None None] MFA Rules not processed for user `53e8bd5b3a0f4c8d9e20e6eb0cfef46a`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:21.675644 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-68d88cd5-f13d-4951-92c0-a29a71454088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:21.752967 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1046/2110] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:05:21 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:05:21.764525 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8894564f-336c-435a-9b22-d64e9e2fbef4 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:21.766274 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-8894564f-336c-435a-9b22-d64e9e2fbef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:21.834454 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8894564f-336c-435a-9b22-d64e9e2fbef4 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:21.838954 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53be122-83b4-4273-9193-db04dce450c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:21.839485 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53be122-83b4-4273-9193-db04dce450c3 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:21.839641 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53be122-83b4-4273-9193-db04dce450c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:21.839993 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53be122-83b4-4273-9193-db04dce450c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:21.899914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53be122-83b4-4273-9193-db04dce450c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:21.901938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53be122-83b4-4273-9193-db04dce450c3 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:21.911069 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1065/2111] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:23.284525 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d88cd5-f13d-4951-92c0-a29a71454088 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:23.285994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-68d88cd5-f13d-4951-92c0-a29a71454088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:23.369391 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d88cd5-f13d-4951-92c0-a29a71454088 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:23.376566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6561884-41bc-40fc-9644-6ef44fdc2eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:23.377440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6561884-41bc-40fc-9644-6ef44fdc2eca admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:23.377704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6561884-41bc-40fc-9644-6ef44fdc2eca admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:23.378544 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6561884-41bc-40fc-9644-6ef44fdc2eca admin admin] PATH_INFO: `/v3/users/8a1e00a787174dafba804a34a3eaf43c` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:23.382713 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6561884-41bc-40fc-9644-6ef44fdc2eca admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a1e00a787174dafba804a34a3eaf43c)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:23.385152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6561884-41bc-40fc-9644-6ef44fdc2eca admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:23.621240 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-b6561884-41bc-40fc-9644-6ef44fdc2eca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a1e00a787174dafba804a34a3eaf43c'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:23.681397 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-b6561884-41bc-40fc-9644-6ef44fdc2eca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a1e00a787174dafba804a34a3eaf43c'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:23.681397 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-b6561884-41bc-40fc-9644-6ef44fdc2eca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a1e00a787174dafba804a34a3eaf43c'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:23.681397 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-b6561884-41bc-40fc-9644-6ef44fdc2eca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a1e00a787174dafba804a34a3eaf43c'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:23.707802 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-b6561884-41bc-40fc-9644-6ef44fdc2eca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a1e00a787174dafba804a34a3eaf43c'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:23.710838 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:23.724503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1047/2112] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:05:23 2022] DELETE /identity/v3/users/8a1e00a787174dafba804a34a3eaf43c => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:23.729074 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab3af5-40f9-4bb9-89d4-ace6be13b95a None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:23.729594 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab3af5-40f9-4bb9-89d4-ace6be13b95a None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:23.729687 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab3af5-40f9-4bb9-89d4-ace6be13b95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:23.750338 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-30ab3af5-40f9-4bb9-89d4-ace6be13b95a None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:23.751592 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-30ab3af5-40f9-4bb9-89d4-ace6be13b95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:23.823081 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1066/2113] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:23.833392 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6561884-41bc-40fc-9644-6ef44fdc2eca admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:23.834891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-b6561884-41bc-40fc-9644-6ef44fdc2eca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:23.896567 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6561884-41bc-40fc-9644-6ef44fdc2eca admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:23.900360 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065de8af-4cd3-4edd-a015-483d0e9f9dfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:23.901673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065de8af-4cd3-4edd-a015-483d0e9f9dfd service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:23.902041 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065de8af-4cd3-4edd-a015-483d0e9f9dfd service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:23.902324 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065de8af-4cd3-4edd-a015-483d0e9f9dfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:23.912334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065de8af-4cd3-4edd-a015-483d0e9f9dfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:23.913777 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065de8af-4cd3-4edd-a015-483d0e9f9dfd service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:23.924346 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1048/2114] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:24.429061 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab3af5-40f9-4bb9-89d4-ace6be13b95a None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:24.429061 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-30ab3af5-40f9-4bb9-89d4-ace6be13b95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:24.443231 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab3af5-40f9-4bb9-89d4-ace6be13b95a None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:24.450269 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da630dbf-6d46-40d4-bc53-2da51990a239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:24.451103 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da630dbf-6d46-40d4-bc53-2da51990a239 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:24.456819 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da630dbf-6d46-40d4-bc53-2da51990a239 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:24.456819 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da630dbf-6d46-40d4-bc53-2da51990a239 admin admin] PATH_INFO: `/v3/projects/e20e3edeebd04d70be3f4116817593b3` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:24.456819 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da630dbf-6d46-40d4-bc53-2da51990a239 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e20e3edeebd04d70be3f4116817593b3)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:24.456819 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da630dbf-6d46-40d4-bc53-2da51990a239 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:24.468020 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:24.642542 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-da630dbf-6d46-40d4-bc53-2da51990a239 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e20e3edeebd04d70be3f4116817593b3', 'request_id': 'req-da630dbf-6d46-40d4-bc53-2da51990a239'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:24.698819 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-da630dbf-6d46-40d4-bc53-2da51990a239 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e20e3edeebd04d70be3f4116817593b3', 'request_id': 'req-da630dbf-6d46-40d4-bc53-2da51990a239'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:24.704765 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-da630dbf-6d46-40d4-bc53-2da51990a239 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e20e3edeebd04d70be3f4116817593b3', 'request_id': 'req-da630dbf-6d46-40d4-bc53-2da51990a239'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:24.718068 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-da630dbf-6d46-40d4-bc53-2da51990a239 admin admin] The token cache is being invalidate because project e20e3edeebd04d70be3f4116817593b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:05:24.718363 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-da630dbf-6d46-40d4-bc53-2da51990a239 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:24.719050 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-da630dbf-6d46-40d4-bc53-2da51990a239 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:24.720608 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1067/2115] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:05:24 2022] DELETE /identity/v3/projects/e20e3edeebd04d70be3f4116817593b3 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:24.725519 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9185016-50b8-40a6-8ca5-e17e16b9b5da None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:24.725995 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9185016-50b8-40a6-8ca5-e17e16b9b5da None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:24.726106 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9185016-50b8-40a6-8ca5-e17e16b9b5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:24.747498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-d9185016-50b8-40a6-8ca5-e17e16b9b5da None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:24.748801 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-d9185016-50b8-40a6-8ca5-e17e16b9b5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:24.816968 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1049/2116] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:05:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:24.826666 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539795fa-d44b-4163-a46c-fad2db7ad6ba None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:24.827218 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539795fa-d44b-4163-a46c-fad2db7ad6ba None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:24.827218 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539795fa-d44b-4163-a46c-fad2db7ad6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:24.849398 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-539795fa-d44b-4163-a46c-fad2db7ad6ba None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:24.850641 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-539795fa-d44b-4163-a46c-fad2db7ad6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:24.859502 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1068/2117] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:05:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:24.864232 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9185016-50b8-40a6-8ca5-e17e16b9b5da None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:24.877972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9185016-50b8-40a6-8ca5-e17e16b9b5da None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:24.884132 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044de6ec-743c-46ba-8e64-cfdf94316081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:24.886534 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044de6ec-743c-46ba-8e64-cfdf94316081 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:24.886818 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044de6ec-743c-46ba-8e64-cfdf94316081 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:24.887098 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044de6ec-743c-46ba-8e64-cfdf94316081 admin admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:24.887760 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044de6ec-743c-46ba-8e64-cfdf94316081 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:05:24.888138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044de6ec-743c-46ba-8e64-cfdf94316081 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:24.889879 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044de6ec-743c-46ba-8e64-cfdf94316081 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:24.904764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1050/2118] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:05:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:05:24.908842 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539795fa-d44b-4163-a46c-fad2db7ad6ba None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:24.920669 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539795fa-d44b-4163-a46c-fad2db7ad6ba None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:24.927247 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3181b-b994-4472-b5c4-f3048f206ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:24.927977 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3181b-b994-4472-b5c4-f3048f206ca9 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:24.928371 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3181b-b994-4472-b5c4-f3048f206ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:24.928720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3181b-b994-4472-b5c4-f3048f206ca9 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:24.929777 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3181b-b994-4472-b5c4-f3048f206ca9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:24.933110 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3181b-b994-4472-b5c4-f3048f206ca9 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:25.011494 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044de6ec-743c-46ba-8e64-cfdf94316081 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:25.013302 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-044de6ec-743c-46ba-8e64-cfdf94316081 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:25.026919 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1069/2119] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:24 2022] POST /identity/v3/projects => generated 391 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:25.031352 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3181b-b994-4472-b5c4-f3048f206ca9 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:25.083855 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044de6ec-743c-46ba-8e64-cfdf94316081 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:25.105385 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3181b-b994-4472-b5c4-f3048f206ca9 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:25.111223 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f30a4-8e17-4382-8064-7800a87653ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:25.111894 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f30a4-8e17-4382-8064-7800a87653ab admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:25.112176 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f30a4-8e17-4382-8064-7800a87653ab admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:25.112615 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f30a4-8e17-4382-8064-7800a87653ab admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:25.113268 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f30a4-8e17-4382-8064-7800a87653ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:25.115613 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f30a4-8e17-4382-8064-7800a87653ab admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:25.130721 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:25.163153 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b9b83-19e2-4072-8a5d-507702bc3286 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:25.164058 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b9b83-19e2-4072-8a5d-507702bc3286 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:25.164557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b9b83-19e2-4072-8a5d-507702bc3286 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:25.165049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b9b83-19e2-4072-8a5d-507702bc3286 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:25.168639 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-4a9b9b83-19e2-4072-8a5d-507702bc3286 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:25.207750 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1070/2120] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:25 2022] POST /identity/v3/users => generated 296 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:05:25.212020 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f30a4-8e17-4382-8064-7800a87653ab admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:25.226470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f30a4-8e17-4382-8064-7800a87653ab admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:25.231571 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f248cba1-dc12-4a78-acc9-59998e2ea870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:25.232175 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248cba1-dc12-4a78-acc9-59998e2ea870 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:25.232449 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248cba1-dc12-4a78-acc9-59998e2ea870 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:25.232719 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248cba1-dc12-4a78-acc9-59998e2ea870 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:25.233362 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f248cba1-dc12-4a78-acc9-59998e2ea870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:25.233708 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b9b83-19e2-4072-8a5d-507702bc3286 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:25.234987 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f248cba1-dc12-4a78-acc9-59998e2ea870 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:25.235179 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b9b83-19e2-4072-8a5d-507702bc3286 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:25.247321 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1071/2121] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:05:25.251584 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1051/2122] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:25.252444 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f248cba1-dc12-4a78-acc9-59998e2ea870 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:25.263894 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f248cba1-dc12-4a78-acc9-59998e2ea870 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:25.269100 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd246c3-7c0b-4d2c-86e9-b8b290ce4ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:25.269714 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd246c3-7c0b-4d2c-86e9-b8b290ce4ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:25.269992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd246c3-7c0b-4d2c-86e9-b8b290ce4ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:25.270263 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd246c3-7c0b-4d2c-86e9-b8b290ce4ca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:25.270945 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd246c3-7c0b-4d2c-86e9-b8b290ce4ca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:25.272655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd246c3-7c0b-4d2c-86e9-b8b290ce4ca1 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:25.284210 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1072/2123] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:25.288709 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b9b83-19e2-4072-8a5d-507702bc3286 service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:25.309448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b9b83-19e2-4072-8a5d-507702bc3286 service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:25.314758 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3e947-d4ee-45aa-8dc6-9cd94bccff71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:25.315507 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3e947-d4ee-45aa-8dc6-9cd94bccff71 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:25.315789 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3e947-d4ee-45aa-8dc6-9cd94bccff71 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:25.316066 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3e947-d4ee-45aa-8dc6-9cd94bccff71 admin admin] PATH_INFO: `/v3/projects/197fab3f7e5a4317b2fecbc6d05b5e88/users/d1caa09a9e5a4f189716c3620969aaf5/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:25.343148 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3e947-d4ee-45aa-8dc6-9cd94bccff71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=197fab3f7e5a4317b2fecbc6d05b5e88, user_id=d1caa09a9e5a4f189716c3620969aaf5, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:25.346448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3e947-d4ee-45aa-8dc6-9cd94bccff71 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:25.441714 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1052/2124] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:25 2022] PUT /identity/v3/projects/197fab3f7e5a4317b2fecbc6d05b5e88/users/d1caa09a9e5a4f189716c3620969aaf5/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:25.453429 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f3d8a-9aab-4b3a-8ce7-233edd98e62c None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:25.453773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f3d8a-9aab-4b3a-8ce7-233edd98e62c None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:25.454052 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f3d8a-9aab-4b3a-8ce7-233edd98e62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:25.484634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-611f3d8a-9aab-4b3a-8ce7-233edd98e62c None None] MFA Rules not processed for user `d1caa09a9e5a4f189716c3620969aaf5`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:25.485848 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-611f3d8a-9aab-4b3a-8ce7-233edd98e62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:25.519622 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:25.561389 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1073/2125] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:05:25 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:25.586118 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3e947-d4ee-45aa-8dc6-9cd94bccff71 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:25.588877 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-dde3e947-d4ee-45aa-8dc6-9cd94bccff71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:25.647964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3e947-d4ee-45aa-8dc6-9cd94bccff71 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:25.652716 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aba8bb-dba5-429a-841f-0dc604ebaa99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:25.653335 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aba8bb-dba5-429a-841f-0dc604ebaa99 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:25.653613 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aba8bb-dba5-429a-841f-0dc604ebaa99 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:25.653888 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aba8bb-dba5-429a-841f-0dc604ebaa99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:25.662948 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aba8bb-dba5-429a-841f-0dc604ebaa99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:25.664412 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aba8bb-dba5-429a-841f-0dc604ebaa99 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:25.673434 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1053/2126] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:25 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:26.836314 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f3d8a-9aab-4b3a-8ce7-233edd98e62c None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:26.845941 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f3d8a-9aab-4b3a-8ce7-233edd98e62c None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:26.849548 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a9499-aa17-433e-8a53-0c5273537c30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:26.850143 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a9499-aa17-433e-8a53-0c5273537c30 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:26.850454 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a9499-aa17-433e-8a53-0c5273537c30 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:26.850728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a9499-aa17-433e-8a53-0c5273537c30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:26.852712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-7f6a9499-aa17-433e-8a53-0c5273537c30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:26.919430 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a9499-aa17-433e-8a53-0c5273537c30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:26.921165 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a9499-aa17-433e-8a53-0c5273537c30 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:26.931248 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1074/2127] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:26 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:35.670149 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=67,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:36.926863 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=45 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:46.374819 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aba8bb-dba5-429a-841f-0dc604ebaa99 service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:46.386765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-d1aba8bb-dba5-429a-841f-0dc604ebaa99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:46.461523 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aba8bb-dba5-429a-841f-0dc604ebaa99 service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:46.467016 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5d7d1-b17a-4541-8001-abecfee96d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:46.467657 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5d7d1-b17a-4541-8001-abecfee96d11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:46.467934 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5d7d1-b17a-4541-8001-abecfee96d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:46.468421 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5d7d1-b17a-4541-8001-abecfee96d11 admin admin] PATH_INFO: `/v3/users/59989a71e17743b9b7d68e269fbb4e88` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:46.469586 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d5d7d1-b17a-4541-8001-abecfee96d11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59989a71e17743b9b7d68e269fbb4e88)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:46.471546 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d5d7d1-b17a-4541-8001-abecfee96d11 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:46.655234 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-02d5d7d1-b17a-4541-8001-abecfee96d11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59989a71e17743b9b7d68e269fbb4e88'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:46.709845 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-02d5d7d1-b17a-4541-8001-abecfee96d11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59989a71e17743b9b7d68e269fbb4e88'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:46.710515 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-02d5d7d1-b17a-4541-8001-abecfee96d11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59989a71e17743b9b7d68e269fbb4e88'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:46.711110 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-02d5d7d1-b17a-4541-8001-abecfee96d11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59989a71e17743b9b7d68e269fbb4e88'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:46.724992 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-02d5d7d1-b17a-4541-8001-abecfee96d11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59989a71e17743b9b7d68e269fbb4e88'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:46.739022 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1054/2128] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:05:46 2022] DELETE /identity/v3/users/59989a71e17743b9b7d68e269fbb4e88 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:05:46.743327 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829e070-4550-4000-a910-a5f67d262273 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:46.744537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829e070-4550-4000-a910-a5f67d262273 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:46.744537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829e070-4550-4000-a910-a5f67d262273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:46.763911 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-5829e070-4550-4000-a910-a5f67d262273 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:46.765223 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-5829e070-4550-4000-a910-a5f67d262273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:46.829845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1075/2129] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:05:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:46.840752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5d7d1-b17a-4541-8001-abecfee96d11 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:46.842343 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-02d5d7d1-b17a-4541-8001-abecfee96d11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:46.904144 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5d7d1-b17a-4541-8001-abecfee96d11 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:46.908069 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacee5ee-5d50-4b82-8ae3-d0cdb7c6219d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:46.908669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacee5ee-5d50-4b82-8ae3-d0cdb7c6219d service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:46.908944 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacee5ee-5d50-4b82-8ae3-d0cdb7c6219d service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:46.909215 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacee5ee-5d50-4b82-8ae3-d0cdb7c6219d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:46.919336 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacee5ee-5d50-4b82-8ae3-d0cdb7c6219d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:46.920765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacee5ee-5d50-4b82-8ae3-d0cdb7c6219d service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:46.931975 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1055/2130] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:47.756377 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829e070-4550-4000-a910-a5f67d262273 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:47.757817 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-5829e070-4550-4000-a910-a5f67d262273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:47.771432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829e070-4550-4000-a910-a5f67d262273 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:47.777020 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d683acb-d128-492f-9652-a96974b6aeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:47.777729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d683acb-d128-492f-9652-a96974b6aeb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:47.777936 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d683acb-d128-492f-9652-a96974b6aeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:47.778209 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d683acb-d128-492f-9652-a96974b6aeb3 admin admin] PATH_INFO: `/v3/projects/fea563d255a74413848ba34c46def500` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:47.779435 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d683acb-d128-492f-9652-a96974b6aeb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fea563d255a74413848ba34c46def500)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:47.781458 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d683acb-d128-492f-9652-a96974b6aeb3 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:47.900002 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-5d683acb-d128-492f-9652-a96974b6aeb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fea563d255a74413848ba34c46def500', 'request_id': 'req-5d683acb-d128-492f-9652-a96974b6aeb3'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:47.982710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-5d683acb-d128-492f-9652-a96974b6aeb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fea563d255a74413848ba34c46def500', 'request_id': 'req-5d683acb-d128-492f-9652-a96974b6aeb3'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:47.989413 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-5d683acb-d128-492f-9652-a96974b6aeb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fea563d255a74413848ba34c46def500', 'request_id': 'req-5d683acb-d128-492f-9652-a96974b6aeb3'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:48.001690 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-5d683acb-d128-492f-9652-a96974b6aeb3 admin admin] The token cache is being invalidate because project fea563d255a74413848ba34c46def500 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:05:48.001981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-5d683acb-d128-492f-9652-a96974b6aeb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:48.002686 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-5d683acb-d128-492f-9652-a96974b6aeb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:48.004158 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1076/2131] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:05:47 2022] DELETE /identity/v3/projects/fea563d255a74413848ba34c46def500 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:48.008351 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacee5ee-5d50-4b82-8ae3-d0cdb7c6219d service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:48.018728 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacee5ee-5d50-4b82-8ae3-d0cdb7c6219d service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:48.039745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:48.082298 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920461b-d5e6-4f7f-8a18-b7b176b1d17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:48.083245 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920461b-d5e6-4f7f-8a18-b7b176b1d17a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:48.083680 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920461b-d5e6-4f7f-8a18-b7b176b1d17a admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:48.084158 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920461b-d5e6-4f7f-8a18-b7b176b1d17a admin admin] PATH_INFO: `/v3/users/b01aa659aca14ba3a677f5d4a8944e98` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:48.085455 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920461b-d5e6-4f7f-8a18-b7b176b1d17a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b01aa659aca14ba3a677f5d4a8944e98)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:48.088091 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920461b-d5e6-4f7f-8a18-b7b176b1d17a admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:48.361518 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-d920461b-d5e6-4f7f-8a18-b7b176b1d17a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b01aa659aca14ba3a677f5d4a8944e98'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:48.428994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-d920461b-d5e6-4f7f-8a18-b7b176b1d17a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b01aa659aca14ba3a677f5d4a8944e98'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:48.431889 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-d920461b-d5e6-4f7f-8a18-b7b176b1d17a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b01aa659aca14ba3a677f5d4a8944e98'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:48.431889 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-d920461b-d5e6-4f7f-8a18-b7b176b1d17a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b01aa659aca14ba3a677f5d4a8944e98'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:48.446223 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-d920461b-d5e6-4f7f-8a18-b7b176b1d17a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b01aa659aca14ba3a677f5d4a8944e98'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:48.461203 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1056/2132] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:05:48 2022] DELETE /identity/v3/users/b01aa659aca14ba3a677f5d4a8944e98 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:48.931775 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d683acb-d128-492f-9652-a96974b6aeb3 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:48.934586 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-5d683acb-d128-492f-9652-a96974b6aeb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:48.959221 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d683acb-d128-492f-9652-a96974b6aeb3 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:49.057481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6239c8d-2d11-4353-af7e-dfe3e3107d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:49.060968 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6239c8d-2d11-4353-af7e-dfe3e3107d48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:49.061410 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6239c8d-2d11-4353-af7e-dfe3e3107d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:49.061821 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6239c8d-2d11-4353-af7e-dfe3e3107d48 admin admin] PATH_INFO: `/v3/projects/5257d9c6b240499eb38d7ea01151d822` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:49.064294 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6239c8d-2d11-4353-af7e-dfe3e3107d48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5257d9c6b240499eb38d7ea01151d822)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:49.066269 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6239c8d-2d11-4353-af7e-dfe3e3107d48 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:49.202958 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:49.221885 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-d6239c8d-2d11-4353-af7e-dfe3e3107d48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5257d9c6b240499eb38d7ea01151d822', 'request_id': 'req-d6239c8d-2d11-4353-af7e-dfe3e3107d48'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:49.280977 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-d6239c8d-2d11-4353-af7e-dfe3e3107d48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5257d9c6b240499eb38d7ea01151d822', 'request_id': 'req-d6239c8d-2d11-4353-af7e-dfe3e3107d48'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:49.287204 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-d6239c8d-2d11-4353-af7e-dfe3e3107d48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5257d9c6b240499eb38d7ea01151d822', 'request_id': 'req-d6239c8d-2d11-4353-af7e-dfe3e3107d48'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:49.300478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-d6239c8d-2d11-4353-af7e-dfe3e3107d48 admin admin] The token cache is being invalidate because project 5257d9c6b240499eb38d7ea01151d822 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:05:49.300852 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-d6239c8d-2d11-4353-af7e-dfe3e3107d48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:49.301502 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-d6239c8d-2d11-4353-af7e-dfe3e3107d48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:49.303078 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1077/2133] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:05:48 2022] DELETE /identity/v3/projects/5257d9c6b240499eb38d7ea01151d822 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:49.307683 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920461b-d5e6-4f7f-8a18-b7b176b1d17a admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:49.314024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-d920461b-d5e6-4f7f-8a18-b7b176b1d17a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:49.323191 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920461b-d5e6-4f7f-8a18-b7b176b1d17a admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:49.381042 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa9e08-b22a-4e86-8668-21f5e5c33e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:49.381671 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa9e08-b22a-4e86-8668-21f5e5c33e7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:49.381948 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa9e08-b22a-4e86-8668-21f5e5c33e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:49.382215 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa9e08-b22a-4e86-8668-21f5e5c33e7b admin admin] PATH_INFO: `/v3/users/53e8bd5b3a0f4c8d9e20e6eb0cfef46a` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:49.383375 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa9e08-b22a-4e86-8668-21f5e5c33e7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=53e8bd5b3a0f4c8d9e20e6eb0cfef46a)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:49.388557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa9e08-b22a-4e86-8668-21f5e5c33e7b admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:49.636523 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:49.644407 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-31aa9e08-b22a-4e86-8668-21f5e5c33e7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53e8bd5b3a0f4c8d9e20e6eb0cfef46a'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:49.731921 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-31aa9e08-b22a-4e86-8668-21f5e5c33e7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53e8bd5b3a0f4c8d9e20e6eb0cfef46a'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:49.732663 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-31aa9e08-b22a-4e86-8668-21f5e5c33e7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53e8bd5b3a0f4c8d9e20e6eb0cfef46a'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:49.733332 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-31aa9e08-b22a-4e86-8668-21f5e5c33e7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53e8bd5b3a0f4c8d9e20e6eb0cfef46a'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:49.754237 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-31aa9e08-b22a-4e86-8668-21f5e5c33e7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53e8bd5b3a0f4c8d9e20e6eb0cfef46a'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:49.773374 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1057/2134] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:05:49 2022] DELETE /identity/v3/users/53e8bd5b3a0f4c8d9e20e6eb0cfef46a => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:50.301457 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6239c8d-2d11-4353-af7e-dfe3e3107d48 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:50.302950 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-d6239c8d-2d11-4353-af7e-dfe3e3107d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:50.316625 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6239c8d-2d11-4353-af7e-dfe3e3107d48 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:50.369164 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa9e08-b22a-4e86-8668-21f5e5c33e7b admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:50.371173 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-31aa9e08-b22a-4e86-8668-21f5e5c33e7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:50.411217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14067f30-c801-4e04-85f9-531291625ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:50.411857 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14067f30-c801-4e04-85f9-531291625ce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:50.412143 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14067f30-c801-4e04-85f9-531291625ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:50.412400 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14067f30-c801-4e04-85f9-531291625ce1 admin admin] PATH_INFO: `/v3/projects/f3fc4d9706794cd4bcb7892122251821` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:50.414354 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14067f30-c801-4e04-85f9-531291625ce1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3fc4d9706794cd4bcb7892122251821)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:50.418632 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14067f30-c801-4e04-85f9-531291625ce1 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:50.453522 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa9e08-b22a-4e86-8668-21f5e5c33e7b admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:50.458413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a42f94-50ef-41c3-940e-53f670872d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:50.459056 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a42f94-50ef-41c3-940e-53f670872d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:50.459334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a42f94-50ef-41c3-940e-53f670872d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:50.459606 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a42f94-50ef-41c3-940e-53f670872d40 admin admin] PATH_INFO: `/v3/users/d1c0bed0663c4afb8c59dd77be320905` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:50.460806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a42f94-50ef-41c3-940e-53f670872d40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1c0bed0663c4afb8c59dd77be320905)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:50.463404 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a42f94-50ef-41c3-940e-53f670872d40 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:50.526439 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-14067f30-c801-4e04-85f9-531291625ce1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3fc4d9706794cd4bcb7892122251821', 'request_id': 'req-14067f30-c801-4e04-85f9-531291625ce1'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:50.606527 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-14067f30-c801-4e04-85f9-531291625ce1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3fc4d9706794cd4bcb7892122251821', 'request_id': 'req-14067f30-c801-4e04-85f9-531291625ce1'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:50.612786 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-14067f30-c801-4e04-85f9-531291625ce1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3fc4d9706794cd4bcb7892122251821', 'request_id': 'req-14067f30-c801-4e04-85f9-531291625ce1'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:50.624849 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-14067f30-c801-4e04-85f9-531291625ce1 admin admin] The token cache is being invalidate because project f3fc4d9706794cd4bcb7892122251821 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:05:50.625137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-14067f30-c801-4e04-85f9-531291625ce1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:50.625833 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-14067f30-c801-4e04-85f9-531291625ce1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:50.627361 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1078/2135] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:05:50 2022] DELETE /identity/v3/projects/f3fc4d9706794cd4bcb7892122251821 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:50.650829 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-45a42f94-50ef-41c3-940e-53f670872d40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1c0bed0663c4afb8c59dd77be320905'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:50.720281 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-45a42f94-50ef-41c3-940e-53f670872d40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1c0bed0663c4afb8c59dd77be320905'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:50.721087 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-45a42f94-50ef-41c3-940e-53f670872d40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1c0bed0663c4afb8c59dd77be320905'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:50.721754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-45a42f94-50ef-41c3-940e-53f670872d40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1c0bed0663c4afb8c59dd77be320905'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:50.741091 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-45a42f94-50ef-41c3-940e-53f670872d40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1c0bed0663c4afb8c59dd77be320905'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:50.757122 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1058/2136] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:05:50 2022] DELETE /identity/v3/users/d1c0bed0663c4afb8c59dd77be320905 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:51.008918 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14067f30-c801-4e04-85f9-531291625ce1 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:51.013354 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-14067f30-c801-4e04-85f9-531291625ce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:51.074112 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:51.092603 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14067f30-c801-4e04-85f9-531291625ce1 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:51.154167 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba298224-b4b9-47af-8ca3-a8b98c3f3332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:51.154910 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba298224-b4b9-47af-8ca3-a8b98c3f3332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:51.155211 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba298224-b4b9-47af-8ca3-a8b98c3f3332 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:51.155524 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba298224-b4b9-47af-8ca3-a8b98c3f3332 admin admin] PATH_INFO: `/v3/projects/37d98454f0d04e9496623da6700329f8` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:51.156816 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba298224-b4b9-47af-8ca3-a8b98c3f3332 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37d98454f0d04e9496623da6700329f8)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:51.159038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba298224-b4b9-47af-8ca3-a8b98c3f3332 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:51.252873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ba298224-b4b9-47af-8ca3-a8b98c3f3332 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37d98454f0d04e9496623da6700329f8', 'request_id': 'req-ba298224-b4b9-47af-8ca3-a8b98c3f3332'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:51.303973 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ba298224-b4b9-47af-8ca3-a8b98c3f3332 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37d98454f0d04e9496623da6700329f8', 'request_id': 'req-ba298224-b4b9-47af-8ca3-a8b98c3f3332'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:51.310597 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ba298224-b4b9-47af-8ca3-a8b98c3f3332 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37d98454f0d04e9496623da6700329f8', 'request_id': 'req-ba298224-b4b9-47af-8ca3-a8b98c3f3332'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:51.321548 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ba298224-b4b9-47af-8ca3-a8b98c3f3332 admin admin] The token cache is being invalidate because project 37d98454f0d04e9496623da6700329f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:05:51.322499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ba298224-b4b9-47af-8ca3-a8b98c3f3332 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:51.323148 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ba298224-b4b9-47af-8ca3-a8b98c3f3332 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:51.324767 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1079/2137] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:05:51 2022] DELETE /identity/v3/projects/37d98454f0d04e9496623da6700329f8 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:51.330640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f838a3f-6aff-498d-8fb7-5976f4eee120 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:51.330922 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f838a3f-6aff-498d-8fb7-5976f4eee120 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:51.331217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f838a3f-6aff-498d-8fb7-5976f4eee120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:51.351887 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-7f838a3f-6aff-498d-8fb7-5976f4eee120 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:51.353933 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-7f838a3f-6aff-498d-8fb7-5976f4eee120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:51.392286 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:51.419957 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1059/2138] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:05:51.427093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f10171-ee67-4692-af63-2c2164ecf78c None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:51.427427 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f10171-ee67-4692-af63-2c2164ecf78c None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:51.427647 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f10171-ee67-4692-af63-2c2164ecf78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:51.450683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-62f10171-ee67-4692-af63-2c2164ecf78c None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:51.451989 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-62f10171-ee67-4692-af63-2c2164ecf78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:51.461152 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1080/2139] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:51.468025 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f838a3f-6aff-498d-8fb7-5976f4eee120 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:51.476408 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f838a3f-6aff-498d-8fb7-5976f4eee120 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:51.482958 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c1ab9-e77f-4da8-9a79-2db23b909ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:51.483662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c1ab9-e77f-4da8-9a79-2db23b909ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:51.483943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c1ab9-e77f-4da8-9a79-2db23b909ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:51.484216 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c1ab9-e77f-4da8-9a79-2db23b909ee5 admin admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:51.484957 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c1ab9-e77f-4da8-9a79-2db23b909ee5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:05:51.485344 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c1ab9-e77f-4da8-9a79-2db23b909ee5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:51.487384 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c1ab9-e77f-4da8-9a79-2db23b909ee5 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:51.500179 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1060/2140] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:05:51.504542 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f10171-ee67-4692-af63-2c2164ecf78c None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:51.514811 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f10171-ee67-4692-af63-2c2164ecf78c None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:51.520111 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6535fa6-13e4-471f-9f34-5c8f5722fae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:51.520727 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6535fa6-13e4-471f-9f34-5c8f5722fae6 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:51.521009 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6535fa6-13e4-471f-9f34-5c8f5722fae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:51.521287 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6535fa6-13e4-471f-9f34-5c8f5722fae6 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:51.521931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6535fa6-13e4-471f-9f34-5c8f5722fae6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:51.523891 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6535fa6-13e4-471f-9f34-5c8f5722fae6 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:51.605680 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1081/2141] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:51 2022] POST /identity/v3/projects => generated 389 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:51.608725 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c1ab9-e77f-4da8-9a79-2db23b909ee5 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:51.619446 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c1ab9-e77f-4da8-9a79-2db23b909ee5 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:51.677387 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d7a4de-b933-46e7-965d-902d39642dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:51.678101 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7a4de-b933-46e7-965d-902d39642dc1 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:51.678382 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7a4de-b933-46e7-965d-902d39642dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:51.678674 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7a4de-b933-46e7-965d-902d39642dc1 admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:51.679452 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d7a4de-b933-46e7-965d-902d39642dc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:51.681965 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d7a4de-b933-46e7-965d-902d39642dc1 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:51.768139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1061/2142] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:51 2022] POST /identity/v3/users => generated 295 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:51.776989 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6535fa6-13e4-471f-9f34-5c8f5722fae6 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:51.787510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6535fa6-13e4-471f-9f34-5c8f5722fae6 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:51.792820 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63d97b-66ab-4e74-9a03-b5a38a59be92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:51.793426 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63d97b-66ab-4e74-9a03-b5a38a59be92 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:51.793704 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63d97b-66ab-4e74-9a03-b5a38a59be92 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:51.793972 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63d97b-66ab-4e74-9a03-b5a38a59be92 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:51.794663 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be63d97b-66ab-4e74-9a03-b5a38a59be92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:51.796257 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be63d97b-66ab-4e74-9a03-b5a38a59be92 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:51.810039 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1082/2143] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:51.810959 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d7a4de-b933-46e7-965d-902d39642dc1 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:51.821380 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d7a4de-b933-46e7-965d-902d39642dc1 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:51.827055 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15799d11-5652-4a94-b634-5c24ff920d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:51.827650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15799d11-5652-4a94-b634-5c24ff920d2f admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:51.827930 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15799d11-5652-4a94-b634-5c24ff920d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:51.828221 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15799d11-5652-4a94-b634-5c24ff920d2f admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:51.829377 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15799d11-5652-4a94-b634-5c24ff920d2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:51.830640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15799d11-5652-4a94-b634-5c24ff920d2f admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:51.842481 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1062/2144] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:51.846732 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63d97b-66ab-4e74-9a03-b5a38a59be92 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:51.857185 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63d97b-66ab-4e74-9a03-b5a38a59be92 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:51.861657 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583272f-a34f-434f-8c05-4990efd097f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:51.862327 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583272f-a34f-434f-8c05-4990efd097f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:51.862641 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583272f-a34f-434f-8c05-4990efd097f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:51.862915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583272f-a34f-434f-8c05-4990efd097f1 admin admin] PATH_INFO: `/v3/projects/644e7abc63304b8d950b786654696f21/users/8ab477a1ab0d4e61b9649b62ee5295a0/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:51.877068 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f583272f-a34f-434f-8c05-4990efd097f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=644e7abc63304b8d950b786654696f21, user_id=8ab477a1ab0d4e61b9649b62ee5295a0, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:51.880149 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f583272f-a34f-434f-8c05-4990efd097f1 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:51.984687 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1083/2145] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:51 2022] PUT /identity/v3/projects/644e7abc63304b8d950b786654696f21/users/8ab477a1ab0d4e61b9649b62ee5295a0/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:51.989977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e152e3-be9c-4c2f-8264-5d6d0e2ab07c None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:51.990267 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e152e3-be9c-4c2f-8264-5d6d0e2ab07c None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:51.990594 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e152e3-be9c-4c2f-8264-5d6d0e2ab07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:52.012795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-e6e152e3-be9c-4c2f-8264-5d6d0e2ab07c None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:52.014011 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-e6e152e3-be9c-4c2f-8264-5d6d0e2ab07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:52.074192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1063/2146] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:52.083809 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583272f-a34f-434f-8c05-4990efd097f1 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:52.085573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-f583272f-a34f-434f-8c05-4990efd097f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:52.108422 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:52.147369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583272f-a34f-434f-8c05-4990efd097f1 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:52.152058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27515625-e3f9-40c5-8f93-dee65c2802a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:52.152692 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27515625-e3f9-40c5-8f93-dee65c2802a3 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:52.152974 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27515625-e3f9-40c5-8f93-dee65c2802a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:52.156294 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27515625-e3f9-40c5-8f93-dee65c2802a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:52.163273 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27515625-e3f9-40c5-8f93-dee65c2802a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:52.164735 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27515625-e3f9-40c5-8f93-dee65c2802a3 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:52.176762 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1084/2147] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:58.447131 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7052d-21cf-4aa2-a9b3-9a512c02e2bf None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:58.447658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7052d-21cf-4aa2-a9b3-9a512c02e2bf None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:58.451512 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7052d-21cf-4aa2-a9b3-9a512c02e2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:58.496435 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-9be7052d-21cf-4aa2-a9b3-9a512c02e2bf None None] MFA Rules not processed for user `8ab477a1ab0d4e61b9649b62ee5295a0`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:58.497675 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-9be7052d-21cf-4aa2-a9b3-9a512c02e2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:58.539867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:58.575352 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1064/2148] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:05:58 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:58.589820 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27515625-e3f9-40c5-8f93-dee65c2802a3 service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:58.591498 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-27515625-e3f9-40c5-8f93-dee65c2802a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:58.673931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27515625-e3f9-40c5-8f93-dee65c2802a3 service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:58.679493 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba0070-7d7c-40d2-ab92-292f331b1f8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:58.680113 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba0070-7d7c-40d2-ab92-292f331b1f8f service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:58.680450 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba0070-7d7c-40d2-ab92-292f331b1f8f service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:58.680726 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba0070-7d7c-40d2-ab92-292f331b1f8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:58.691677 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ba0070-7d7c-40d2-ab92-292f331b1f8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:58.693650 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ba0070-7d7c-40d2-ab92-292f331b1f8f service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:58.710607 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1085/2149] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:58 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:01.521189 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1a541-2cf1-4fa6-ad2a-2081e79e47c9 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:01.522647 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1a541-2cf1-4fa6-ad2a-2081e79e47c9 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:01.524853 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1a541-2cf1-4fa6-ad2a-2081e79e47c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:01.554944 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-81f1a541-2cf1-4fa6-ad2a-2081e79e47c9 None None] MFA Rules not processed for user `8ab477a1ab0d4e61b9649b62ee5295a0`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:01.557491 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-81f1a541-2cf1-4fa6-ad2a-2081e79e47c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:01.565019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1065/2150] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:06:01 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:04.039203 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba0070-7d7c-40d2-ab92-292f331b1f8f service nova] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:04.041420 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-d9ba0070-7d7c-40d2-ab92-292f331b1f8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:04.055338 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba0070-7d7c-40d2-ab92-292f331b1f8f service nova] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:04.060377 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9946d-03c2-4e9b-a96f-35c0c35a0833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:04.061012 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9946d-03c2-4e9b-a96f-35c0c35a0833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:04.061462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9946d-03c2-4e9b-a96f-35c0c35a0833 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:04.064241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9946d-03c2-4e9b-a96f-35c0c35a0833 admin admin] PATH_INFO: `/v3/users/d1caa09a9e5a4f189716c3620969aaf5` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:04.064241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae9946d-03c2-4e9b-a96f-35c0c35a0833 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1caa09a9e5a4f189716c3620969aaf5)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:04.064870 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae9946d-03c2-4e9b-a96f-35c0c35a0833 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:04.239330 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-6ae9946d-03c2-4e9b-a96f-35c0c35a0833 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1caa09a9e5a4f189716c3620969aaf5'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:04.252263 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-6ae9946d-03c2-4e9b-a96f-35c0c35a0833 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1caa09a9e5a4f189716c3620969aaf5'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:04.261571 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-6ae9946d-03c2-4e9b-a96f-35c0c35a0833 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1caa09a9e5a4f189716c3620969aaf5'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:04.338710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-6ae9946d-03c2-4e9b-a96f-35c0c35a0833 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1caa09a9e5a4f189716c3620969aaf5'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:04.339346 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-6ae9946d-03c2-4e9b-a96f-35c0c35a0833 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1caa09a9e5a4f189716c3620969aaf5'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:04.345986 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1086/2151] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:06:04 2022] DELETE /identity/v3/users/d1caa09a9e5a4f189716c3620969aaf5 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:04.350703 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ceece6-482f-4bac-939f-3ad370719f92 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:04.351212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ceece6-482f-4bac-939f-3ad370719f92 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:04.351307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ceece6-482f-4bac-939f-3ad370719f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:04.371824 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-97ceece6-482f-4bac-939f-3ad370719f92 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:04.372888 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-97ceece6-482f-4bac-939f-3ad370719f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:04.443489 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1066/2152] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:04.453878 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9946d-03c2-4e9b-a96f-35c0c35a0833 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:04.455409 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-6ae9946d-03c2-4e9b-a96f-35c0c35a0833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:04.477430 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:04.516995 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9946d-03c2-4e9b-a96f-35c0c35a0833 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:04.521879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0575e53-6865-4fc9-b91a-55b5a6ab896f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:04.521879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0575e53-6865-4fc9-b91a-55b5a6ab896f service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:04.522137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0575e53-6865-4fc9-b91a-55b5a6ab896f service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:04.522434 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0575e53-6865-4fc9-b91a-55b5a6ab896f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:04.532741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0575e53-6865-4fc9-b91a-55b5a6ab896f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:04.534233 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0575e53-6865-4fc9-b91a-55b5a6ab896f service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:04.544676 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1087/2153] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:05.026890 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ceece6-482f-4bac-939f-3ad370719f92 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:05.029791 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-97ceece6-482f-4bac-939f-3ad370719f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:05.043265 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ceece6-482f-4bac-939f-3ad370719f92 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:05.048288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510fd00f-6883-44d6-b188-0da914ca2d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:05.048956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fd00f-6883-44d6-b188-0da914ca2d1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:05.051063 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fd00f-6883-44d6-b188-0da914ca2d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:05.051063 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fd00f-6883-44d6-b188-0da914ca2d1b admin admin] PATH_INFO: `/v3/projects/197fab3f7e5a4317b2fecbc6d05b5e88` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:05.051063 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510fd00f-6883-44d6-b188-0da914ca2d1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=197fab3f7e5a4317b2fecbc6d05b5e88)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:05.052845 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510fd00f-6883-44d6-b188-0da914ca2d1b admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:05.202236 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-510fd00f-6883-44d6-b188-0da914ca2d1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '197fab3f7e5a4317b2fecbc6d05b5e88', 'request_id': 'req-510fd00f-6883-44d6-b188-0da914ca2d1b'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:05.211375 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-510fd00f-6883-44d6-b188-0da914ca2d1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '197fab3f7e5a4317b2fecbc6d05b5e88', 'request_id': 'req-510fd00f-6883-44d6-b188-0da914ca2d1b'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:05.218427 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-510fd00f-6883-44d6-b188-0da914ca2d1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '197fab3f7e5a4317b2fecbc6d05b5e88', 'request_id': 'req-510fd00f-6883-44d6-b188-0da914ca2d1b'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:05.312165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-510fd00f-6883-44d6-b188-0da914ca2d1b admin admin] The token cache is being invalidate because project 197fab3f7e5a4317b2fecbc6d05b5e88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:06:05.312499 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-510fd00f-6883-44d6-b188-0da914ca2d1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:05.313179 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-510fd00f-6883-44d6-b188-0da914ca2d1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:05.315226 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1067/2154] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:06:05 2022] DELETE /identity/v3/projects/197fab3f7e5a4317b2fecbc6d05b5e88 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:13.859572 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0575e53-6865-4fc9-b91a-55b5a6ab896f service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:13.862815 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-e0575e53-6865-4fc9-b91a-55b5a6ab896f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:13.875885 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0575e53-6865-4fc9-b91a-55b5a6ab896f service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:13.945768 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e347fec-0bde-47d3-89e9-fd1afade232c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:13.946712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e347fec-0bde-47d3-89e9-fd1afade232c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:13.947001 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e347fec-0bde-47d3-89e9-fd1afade232c admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:13.947276 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e347fec-0bde-47d3-89e9-fd1afade232c admin admin] PATH_INFO: `/v3/users/8cd414ad803847b7837efc2348c8d848` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:13.948586 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e347fec-0bde-47d3-89e9-fd1afade232c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cd414ad803847b7837efc2348c8d848)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:13.951008 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e347fec-0bde-47d3-89e9-fd1afade232c admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:14.112869 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-7e347fec-0bde-47d3-89e9-fd1afade232c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cd414ad803847b7837efc2348c8d848'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:14.127481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-7e347fec-0bde-47d3-89e9-fd1afade232c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cd414ad803847b7837efc2348c8d848'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:14.135640 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-7e347fec-0bde-47d3-89e9-fd1afade232c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cd414ad803847b7837efc2348c8d848'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:14.206470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-7e347fec-0bde-47d3-89e9-fd1afade232c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cd414ad803847b7837efc2348c8d848'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:14.207760 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-7e347fec-0bde-47d3-89e9-fd1afade232c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cd414ad803847b7837efc2348c8d848'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:14.217221 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1088/2155] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:06:13 2022] DELETE /identity/v3/users/8cd414ad803847b7837efc2348c8d848 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:14.222031 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04508e6-2186-42f5-a77b-82d955a2b13b None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:14.227771 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04508e6-2186-42f5-a77b-82d955a2b13b None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:14.227771 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04508e6-2186-42f5-a77b-82d955a2b13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:14.243937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-b04508e6-2186-42f5-a77b-82d955a2b13b None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:14.245242 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-b04508e6-2186-42f5-a77b-82d955a2b13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:14.293800 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:14.305861 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1068/2156] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:14.315507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e347fec-0bde-47d3-89e9-fd1afade232c admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:14.316936 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-7e347fec-0bde-47d3-89e9-fd1afade232c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:14.361298 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:14.374766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e347fec-0bde-47d3-89e9-fd1afade232c admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:14.378442 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7618e-05b7-4fb9-95a5-b28f46328018 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:14.379042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7618e-05b7-4fb9-95a5-b28f46328018 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:14.379319 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7618e-05b7-4fb9-95a5-b28f46328018 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:14.379591 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7618e-05b7-4fb9-95a5-b28f46328018 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:14.389381 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac7618e-05b7-4fb9-95a5-b28f46328018 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:14.393738 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac7618e-05b7-4fb9-95a5-b28f46328018 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:14.400812 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1089/2157] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:06:14.803062 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04508e6-2186-42f5-a77b-82d955a2b13b None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:14.804484 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-b04508e6-2186-42f5-a77b-82d955a2b13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:14.820033 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04508e6-2186-42f5-a77b-82d955a2b13b None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:14.826447 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997cab1-665c-4649-9d64-9593af7d0470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:14.827190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997cab1-665c-4649-9d64-9593af7d0470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:14.827363 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997cab1-665c-4649-9d64-9593af7d0470 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:14.827676 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997cab1-665c-4649-9d64-9593af7d0470 admin admin] PATH_INFO: `/v3/projects/b9f47c1525634bc3b21a807b43552d95` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:14.828868 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997cab1-665c-4649-9d64-9593af7d0470 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9f47c1525634bc3b21a807b43552d95)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:14.830818 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997cab1-665c-4649-9d64-9593af7d0470 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:14.936427 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2997cab1-665c-4649-9d64-9593af7d0470 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9f47c1525634bc3b21a807b43552d95', 'request_id': 'req-2997cab1-665c-4649-9d64-9593af7d0470'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:14.942997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2997cab1-665c-4649-9d64-9593af7d0470 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9f47c1525634bc3b21a807b43552d95', 'request_id': 'req-2997cab1-665c-4649-9d64-9593af7d0470'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:14.949468 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2997cab1-665c-4649-9d64-9593af7d0470 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9f47c1525634bc3b21a807b43552d95', 'request_id': 'req-2997cab1-665c-4649-9d64-9593af7d0470'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:15.051905 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2997cab1-665c-4649-9d64-9593af7d0470 admin admin] The token cache is being invalidate because project b9f47c1525634bc3b21a807b43552d95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:06:15.052104 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2997cab1-665c-4649-9d64-9593af7d0470 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:15.052718 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2997cab1-665c-4649-9d64-9593af7d0470 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:15.054221 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1069/2158] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:06:14 2022] DELETE /identity/v3/projects/b9f47c1525634bc3b21a807b43552d95 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:15.059970 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee042fe-68e1-44ad-a563-467322d41eac None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:15.060459 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee042fe-68e1-44ad-a563-467322d41eac None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:15.060575 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee042fe-68e1-44ad-a563-467322d41eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:15.080778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-6ee042fe-68e1-44ad-a563-467322d41eac None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:15.081982 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-6ee042fe-68e1-44ad-a563-467322d41eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:15.151995 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1090/2159] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:15.160096 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0ead6-a8ff-4a10-ba65-aa37eea8635d None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:15.160377 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0ead6-a8ff-4a10-ba65-aa37eea8635d None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:15.160649 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0ead6-a8ff-4a10-ba65-aa37eea8635d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:15.183420 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-2dd0ead6-a8ff-4a10-ba65-aa37eea8635d None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:15.187540 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-2dd0ead6-a8ff-4a10-ba65-aa37eea8635d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:15.195218 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1070/2160] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:15.199896 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee042fe-68e1-44ad-a563-467322d41eac None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:15.212422 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee042fe-68e1-44ad-a563-467322d41eac None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:15.228835 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820080dd-1cbc-449d-868d-bb05c73018cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:15.229522 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820080dd-1cbc-449d-868d-bb05c73018cf admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:15.229803 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820080dd-1cbc-449d-868d-bb05c73018cf admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:15.230074 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820080dd-1cbc-449d-868d-bb05c73018cf admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:15.230763 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820080dd-1cbc-449d-868d-bb05c73018cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:06:15.231066 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820080dd-1cbc-449d-868d-bb05c73018cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:15.232701 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820080dd-1cbc-449d-868d-bb05c73018cf admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:15.249728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1091/2161] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:06:15.251335 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd0ead6-a8ff-4a10-ba65-aa37eea8635d None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:15.261450 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd0ead6-a8ff-4a10-ba65-aa37eea8635d None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:15.266952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae696b45-e44f-4095-8a4d-2bc252487789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:15.267557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae696b45-e44f-4095-8a4d-2bc252487789 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:15.267891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae696b45-e44f-4095-8a4d-2bc252487789 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:15.268294 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae696b45-e44f-4095-8a4d-2bc252487789 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:15.268944 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae696b45-e44f-4095-8a4d-2bc252487789 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:15.270895 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae696b45-e44f-4095-8a4d-2bc252487789 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:15.385628 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1071/2162] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:06:15 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:06:15.389953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820080dd-1cbc-449d-868d-bb05c73018cf admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:15.400428 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820080dd-1cbc-449d-868d-bb05c73018cf admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:15.457348 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c10c46-5b68-4cc5-9477-4dc9c3e35174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:15.457955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c10c46-5b68-4cc5-9477-4dc9c3e35174 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:15.458239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c10c46-5b68-4cc5-9477-4dc9c3e35174 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:15.458562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c10c46-5b68-4cc5-9477-4dc9c3e35174 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:15.459206 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c10c46-5b68-4cc5-9477-4dc9c3e35174 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:15.461133 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c10c46-5b68-4cc5-9477-4dc9c3e35174 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:15.555975 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1092/2163] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:06:15 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:15.559538 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae696b45-e44f-4095-8a4d-2bc252487789 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:15.569842 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae696b45-e44f-4095-8a4d-2bc252487789 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:15.575261 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3628598-0bda-41a2-bd19-082cc479bc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:15.575857 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3628598-0bda-41a2-bd19-082cc479bc87 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:15.576131 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3628598-0bda-41a2-bd19-082cc479bc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:15.576441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3628598-0bda-41a2-bd19-082cc479bc87 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:15.577082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3628598-0bda-41a2-bd19-082cc479bc87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:15.578719 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3628598-0bda-41a2-bd19-082cc479bc87 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:15.589301 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1072/2164] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:15.593487 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c10c46-5b68-4cc5-9477-4dc9c3e35174 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:15.603920 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c10c46-5b68-4cc5-9477-4dc9c3e35174 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:15.608550 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c2953-891d-464b-a1d1-699b51b2ddd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:15.609138 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c2953-891d-464b-a1d1-699b51b2ddd9 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:15.609417 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c2953-891d-464b-a1d1-699b51b2ddd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:15.609680 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c2953-891d-464b-a1d1-699b51b2ddd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:15.610344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c2953-891d-464b-a1d1-699b51b2ddd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:15.611943 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c2953-891d-464b-a1d1-699b51b2ddd9 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:15.621822 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1093/2165] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:15.626005 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3628598-0bda-41a2-bd19-082cc479bc87 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:15.636689 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3628598-0bda-41a2-bd19-082cc479bc87 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:15.641259 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80314ad-273a-469c-894e-0f9c94e1cc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:15.641923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80314ad-273a-469c-894e-0f9c94e1cc5c admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:15.642194 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80314ad-273a-469c-894e-0f9c94e1cc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:15.642491 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80314ad-273a-469c-894e-0f9c94e1cc5c admin admin] PATH_INFO: `/v3/projects/302a0aa9b9174dc7ac0b8b13423b2784/users/b37dec171a1043ad862870f09bfc92e5/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:15.658739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80314ad-273a-469c-894e-0f9c94e1cc5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=302a0aa9b9174dc7ac0b8b13423b2784, user_id=b37dec171a1043ad862870f09bfc92e5, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:15.661414 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80314ad-273a-469c-894e-0f9c94e1cc5c admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:15.727156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1073/2166] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:15 2022] PUT /identity/v3/projects/302a0aa9b9174dc7ac0b8b13423b2784/users/b37dec171a1043ad862870f09bfc92e5/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:06:15.738839 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ffa60-e256-4d7c-9b91-6058ed1e02c0 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:15.741730 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ffa60-e256-4d7c-9b91-6058ed1e02c0 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:15.741730 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ffa60-e256-4d7c-9b91-6058ed1e02c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:15.768251 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-ec1ffa60-e256-4d7c-9b91-6058ed1e02c0 None None] MFA Rules not processed for user `b37dec171a1043ad862870f09bfc92e5`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:15.770410 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-ec1ffa60-e256-4d7c-9b91-6058ed1e02c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:15.809644 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:15.835572 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1094/2167] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:06:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:15.847046 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80314ad-273a-469c-894e-0f9c94e1cc5c admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:15.848894 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-b80314ad-273a-469c-894e-0f9c94e1cc5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:15.906606 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80314ad-273a-469c-894e-0f9c94e1cc5c admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:15.910293 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c01048-d994-48ff-88aa-1f8f3dd91990 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:15.910925 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c01048-d994-48ff-88aa-1f8f3dd91990 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:15.911206 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c01048-d994-48ff-88aa-1f8f3dd91990 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:15.911479 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c01048-d994-48ff-88aa-1f8f3dd91990 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:15.919856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c01048-d994-48ff-88aa-1f8f3dd91990 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:15.921281 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c01048-d994-48ff-88aa-1f8f3dd91990 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:15.930020 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1074/2168] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:25.868287 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=10 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:25.926099 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:27.361706 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ffa60-e256-4d7c-9b91-6058ed1e02c0 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:27.363188 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-ec1ffa60-e256-4d7c-9b91-6058ed1e02c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:27.440032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ffa60-e256-4d7c-9b91-6058ed1e02c0 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:27.445231 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5c272-423f-4db6-ba56-3de649685ad1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3db46a4b4e74cc0bb2a7bd6f0e99105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:27.445871 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5c272-423f-4db6-ba56-3de649685ad1 service placement] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:27.446148 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5c272-423f-4db6-ba56-3de649685ad1 service placement] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:27.446469 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5c272-423f-4db6-ba56-3de649685ad1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:27.448266 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-cdf5c272-423f-4db6-ba56-3de649685ad1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:27.460381 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5c272-423f-4db6-ba56-3de649685ad1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:27.461980 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5c272-423f-4db6-ba56-3de649685ad1 service placement] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:27.472642 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1095/2169] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:33.466184 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c01048-d994-48ff-88aa-1f8f3dd91990 service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:33.532123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c01048-d994-48ff-88aa-1f8f3dd91990 service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:33.536607 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20fce6-28cd-4861-8147-4947aaed52bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:33.537231 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20fce6-28cd-4861-8147-4947aaed52bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:33.537507 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20fce6-28cd-4861-8147-4947aaed52bc admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:33.537778 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20fce6-28cd-4861-8147-4947aaed52bc admin admin] PATH_INFO: `/v3/users/b37dec171a1043ad862870f09bfc92e5` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:33.538927 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20fce6-28cd-4861-8147-4947aaed52bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b37dec171a1043ad862870f09bfc92e5)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:33.540839 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20fce6-28cd-4861-8147-4947aaed52bc admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:33.779696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-9d20fce6-28cd-4861-8147-4947aaed52bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b37dec171a1043ad862870f09bfc92e5'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:33.845425 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-9d20fce6-28cd-4861-8147-4947aaed52bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b37dec171a1043ad862870f09bfc92e5'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:33.845994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-9d20fce6-28cd-4861-8147-4947aaed52bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b37dec171a1043ad862870f09bfc92e5'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:33.846626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-9d20fce6-28cd-4861-8147-4947aaed52bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b37dec171a1043ad862870f09bfc92e5'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:33.859681 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-9d20fce6-28cd-4861-8147-4947aaed52bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b37dec171a1043ad862870f09bfc92e5'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:33.872353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1075/2170] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:06:33 2022] DELETE /identity/v3/users/b37dec171a1043ad862870f09bfc92e5 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:33.876879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae437f4b-d3c0-4910-a0e8-e19519b6ea74 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:33.877351 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae437f4b-d3c0-4910-a0e8-e19519b6ea74 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:33.877455 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae437f4b-d3c0-4910-a0e8-e19519b6ea74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:33.899917 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-ae437f4b-d3c0-4910-a0e8-e19519b6ea74 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:33.901142 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-ae437f4b-d3c0-4910-a0e8-e19519b6ea74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:33.964432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1096/2171] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:33.974441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20fce6-28cd-4861-8147-4947aaed52bc admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:33.975905 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-9d20fce6-28cd-4861-8147-4947aaed52bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:34.035100 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20fce6-28cd-4861-8147-4947aaed52bc admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:34.038638 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a133a9e-5292-4efc-b60b-42c0f90aa3f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:34.039285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a133a9e-5292-4efc-b60b-42c0f90aa3f6 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:34.039563 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a133a9e-5292-4efc-b60b-42c0f90aa3f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:34.039834 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a133a9e-5292-4efc-b60b-42c0f90aa3f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:34.051967 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a133a9e-5292-4efc-b60b-42c0f90aa3f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:34.053393 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a133a9e-5292-4efc-b60b-42c0f90aa3f6 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:34.064419 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1076/2172] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:34.527826 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae437f4b-d3c0-4910-a0e8-e19519b6ea74 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:34.529591 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-ae437f4b-d3c0-4910-a0e8-e19519b6ea74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:34.547808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae437f4b-d3c0-4910-a0e8-e19519b6ea74 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:34.548437 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5eb0a0-cb22-4080-807a-a0e8f224ea9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:34.548843 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5eb0a0-cb22-4080-807a-a0e8f224ea9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:34.549099 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5eb0a0-cb22-4080-807a-a0e8f224ea9a admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:34.549371 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5eb0a0-cb22-4080-807a-a0e8f224ea9a admin admin] PATH_INFO: `/v3/projects/302a0aa9b9174dc7ac0b8b13423b2784` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:34.550544 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5eb0a0-cb22-4080-807a-a0e8f224ea9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=302a0aa9b9174dc7ac0b8b13423b2784)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:34.552580 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5eb0a0-cb22-4080-807a-a0e8f224ea9a admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:34.697450 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ec5eb0a0-cb22-4080-807a-a0e8f224ea9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '302a0aa9b9174dc7ac0b8b13423b2784', 'request_id': 'req-ec5eb0a0-cb22-4080-807a-a0e8f224ea9a'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:34.765590 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ec5eb0a0-cb22-4080-807a-a0e8f224ea9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '302a0aa9b9174dc7ac0b8b13423b2784', 'request_id': 'req-ec5eb0a0-cb22-4080-807a-a0e8f224ea9a'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:34.771635 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ec5eb0a0-cb22-4080-807a-a0e8f224ea9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '302a0aa9b9174dc7ac0b8b13423b2784', 'request_id': 'req-ec5eb0a0-cb22-4080-807a-a0e8f224ea9a'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:34.783186 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ec5eb0a0-cb22-4080-807a-a0e8f224ea9a admin admin] The token cache is being invalidate because project 302a0aa9b9174dc7ac0b8b13423b2784 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:06:34.783469 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ec5eb0a0-cb22-4080-807a-a0e8f224ea9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:34.784134 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ec5eb0a0-cb22-4080-807a-a0e8f224ea9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:34.785603 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1097/2173] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:06:34 2022] DELETE /identity/v3/projects/302a0aa9b9174dc7ac0b8b13423b2784 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:34.791973 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5f247-4e11-4252-a9f5-e85e0569c6b3 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:34.792445 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5f247-4e11-4252-a9f5-e85e0569c6b3 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:34.792539 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5f247-4e11-4252-a9f5-e85e0569c6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:34.812315 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-bbd5f247-4e11-4252-a9f5-e85e0569c6b3 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:34.813424 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-bbd5f247-4e11-4252-a9f5-e85e0569c6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:34.830901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:34.873601 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1077/2174] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:06:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:34.881723 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35979fae-45d7-4f2d-a2b7-8bcf0ec8c97c None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:34.882174 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35979fae-45d7-4f2d-a2b7-8bcf0ec8c97c None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:34.882264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35979fae-45d7-4f2d-a2b7-8bcf0ec8c97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:34.903088 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-35979fae-45d7-4f2d-a2b7-8bcf0ec8c97c None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:34.904290 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-35979fae-45d7-4f2d-a2b7-8bcf0ec8c97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:34.912351 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1098/2175] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:06:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:06:34.916456 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5f247-4e11-4252-a9f5-e85e0569c6b3 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:34.927130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5f247-4e11-4252-a9f5-e85e0569c6b3 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:34.931876 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1b1a7-7aef-4901-91cb-df9f5f135bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:34.932530 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1b1a7-7aef-4901-91cb-df9f5f135bf5 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:34.932982 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1b1a7-7aef-4901-91cb-df9f5f135bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:34.933353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1b1a7-7aef-4901-91cb-df9f5f135bf5 admin admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:34.934030 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1b1a7-7aef-4901-91cb-df9f5f135bf5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:06:34.934333 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1b1a7-7aef-4901-91cb-df9f5f135bf5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:34.935972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1b1a7-7aef-4901-91cb-df9f5f135bf5 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:34.948118 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1078/2176] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:06:34.952915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35979fae-45d7-4f2d-a2b7-8bcf0ec8c97c None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:34.962083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35979fae-45d7-4f2d-a2b7-8bcf0ec8c97c None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:34.967684 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f446a-c717-4fa3-a832-7e00fb47fff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:34.968278 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f446a-c717-4fa3-a832-7e00fb47fff2 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:34.968557 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f446a-c717-4fa3-a832-7e00fb47fff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:34.968826 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f446a-c717-4fa3-a832-7e00fb47fff2 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:34.969471 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7f446a-c717-4fa3-a832-7e00fb47fff2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:34.971437 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7f446a-c717-4fa3-a832-7e00fb47fff2 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:34.976072 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:35.069040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1099/2177] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:06:34 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:35.073162 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1b1a7-7aef-4901-91cb-df9f5f135bf5 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:35.084310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1b1a7-7aef-4901-91cb-df9f5f135bf5 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:35.140568 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566be5f-92b0-4a6f-955a-e630ee70946d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:35.141177 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566be5f-92b0-4a6f-955a-e630ee70946d admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:35.141452 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566be5f-92b0-4a6f-955a-e630ee70946d admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:35.141719 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566be5f-92b0-4a6f-955a-e630ee70946d admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:35.142349 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e566be5f-92b0-4a6f-955a-e630ee70946d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:35.144275 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e566be5f-92b0-4a6f-955a-e630ee70946d admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:35.271179 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1079/2178] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:06:35 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:35.275469 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f446a-c717-4fa3-a832-7e00fb47fff2 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:35.285838 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f446a-c717-4fa3-a832-7e00fb47fff2 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:35.290418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bafb3bb-551c-4feb-8064-3d33fe41a5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:35.291021 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafb3bb-551c-4feb-8064-3d33fe41a5b6 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:35.291297 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafb3bb-551c-4feb-8064-3d33fe41a5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:35.291566 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafb3bb-551c-4feb-8064-3d33fe41a5b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:35.292205 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bafb3bb-551c-4feb-8064-3d33fe41a5b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:35.294089 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bafb3bb-551c-4feb-8064-3d33fe41a5b6 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:35.304237 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1100/2179] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:35.308811 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566be5f-92b0-4a6f-955a-e630ee70946d admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:35.318911 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566be5f-92b0-4a6f-955a-e630ee70946d admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:35.324404 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d51d84e-d17b-4c14-b284-1052d0964a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:35.324963 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51d84e-d17b-4c14-b284-1052d0964a53 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:35.325254 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51d84e-d17b-4c14-b284-1052d0964a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:35.325525 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51d84e-d17b-4c14-b284-1052d0964a53 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:35.326181 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d51d84e-d17b-4c14-b284-1052d0964a53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:35.327817 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d51d84e-d17b-4c14-b284-1052d0964a53 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:35.338769 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1080/2180] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:35.343338 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bafb3bb-551c-4feb-8064-3d33fe41a5b6 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:35.353697 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bafb3bb-551c-4feb-8064-3d33fe41a5b6 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:35.358706 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561e0de-da99-4b59-bce3-08f643142a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:35.359052 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561e0de-da99-4b59-bce3-08f643142a85 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:35.359332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561e0de-da99-4b59-bce3-08f643142a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:35.359603 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561e0de-da99-4b59-bce3-08f643142a85 admin admin] PATH_INFO: `/v3/projects/a8f4d03f17c94f0b8a22bd2d2fbb793b/users/82e5851d5c6a4a4c9168a64eb922a4d4/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:35.375489 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561e0de-da99-4b59-bce3-08f643142a85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8f4d03f17c94f0b8a22bd2d2fbb793b, user_id=82e5851d5c6a4a4c9168a64eb922a4d4, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:35.378294 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561e0de-da99-4b59-bce3-08f643142a85 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:35.481180 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1101/2181] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:35 2022] PUT /identity/v3/projects/a8f4d03f17c94f0b8a22bd2d2fbb793b/users/82e5851d5c6a4a4c9168a64eb922a4d4/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:35.486210 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb1c7a-be0e-4adf-a79f-258afdd6c2c0 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:35.486630 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb1c7a-be0e-4adf-a79f-258afdd6c2c0 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:35.486906 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb1c7a-be0e-4adf-a79f-258afdd6c2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:35.507403 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-06eb1c7a-be0e-4adf-a79f-258afdd6c2c0 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:35.508649 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-06eb1c7a-be0e-4adf-a79f-258afdd6c2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:35.568242 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1081/2182] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:35.577946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561e0de-da99-4b59-bce3-08f643142a85 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:35.579609 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-6561e0de-da99-4b59-bce3-08f643142a85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:35.637010 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561e0de-da99-4b59-bce3-08f643142a85 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:35.640629 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf7e504-36d8-48f4-b8ee-2e3d385b6e28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:35.641227 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7e504-36d8-48f4-b8ee-2e3d385b6e28 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:35.641508 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7e504-36d8-48f4-b8ee-2e3d385b6e28 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:35.641774 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7e504-36d8-48f4-b8ee-2e3d385b6e28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:35.650705 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf7e504-36d8-48f4-b8ee-2e3d385b6e28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:35.652239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf7e504-36d8-48f4-b8ee-2e3d385b6e28 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:35.661475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1102/2183] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:41.570167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f151dc-2ab8-4242-8b9b-3a408b7b749b None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:41.570736 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f151dc-2ab8-4242-8b9b-3a408b7b749b None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:41.570736 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f151dc-2ab8-4242-8b9b-3a408b7b749b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:41.607467 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-66f151dc-2ab8-4242-8b9b-3a408b7b749b None None] MFA Rules not processed for user `82e5851d5c6a4a4c9168a64eb922a4d4`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:41.608617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-66f151dc-2ab8-4242-8b9b-3a408b7b749b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:41.626221 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:41.686373 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1082/2184] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:06:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:41.690652 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf7e504-36d8-48f4-b8ee-2e3d385b6e28 service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:41.705798 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf7e504-36d8-48f4-b8ee-2e3d385b6e28 service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:41.711568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf5a4e-78a6-4476-aabb-6a9eac06d333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:41.712299 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf5a4e-78a6-4476-aabb-6a9eac06d333 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:41.712645 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf5a4e-78a6-4476-aabb-6a9eac06d333 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:41.713009 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf5a4e-78a6-4476-aabb-6a9eac06d333 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:41.713649 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaf5a4e-78a6-4476-aabb-6a9eac06d333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:41.716349 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaf5a4e-78a6-4476-aabb-6a9eac06d333 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:41.835960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1103/2185] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:06:41 2022] POST /identity/v3/projects => generated 385 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:06:41.840393 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f151dc-2ab8-4242-8b9b-3a408b7b749b None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:41.851223 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f151dc-2ab8-4242-8b9b-3a408b7b749b None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:41.912086 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c169c6-9fd1-4143-b251-945071281bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:41.912712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c169c6-9fd1-4143-b251-945071281bc1 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:41.913306 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c169c6-9fd1-4143-b251-945071281bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:41.913725 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c169c6-9fd1-4143-b251-945071281bc1 admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:41.914365 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c169c6-9fd1-4143-b251-945071281bc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:41.918565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c169c6-9fd1-4143-b251-945071281bc1 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:42.010112 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1083/2186] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:06:41 2022] POST /identity/v3/users => generated 299 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:42.012764 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf5a4e-78a6-4476-aabb-6a9eac06d333 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:42.023841 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf5a4e-78a6-4476-aabb-6a9eac06d333 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:42.029553 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61438398-7ebf-4169-ad80-b3af97b1c462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:42.033255 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61438398-7ebf-4169-ad80-b3af97b1c462 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:42.033255 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61438398-7ebf-4169-ad80-b3af97b1c462 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:42.033255 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61438398-7ebf-4169-ad80-b3af97b1c462 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:42.033255 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61438398-7ebf-4169-ad80-b3af97b1c462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:42.033255 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61438398-7ebf-4169-ad80-b3af97b1c462 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:42.044809 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1104/2187] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:42.049162 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c169c6-9fd1-4143-b251-945071281bc1 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:42.060897 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c169c6-9fd1-4143-b251-945071281bc1 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:42.066042 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b5967-0251-4ede-9737-747e862fb4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:42.066748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b5967-0251-4ede-9737-747e862fb4fd admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:42.067055 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b5967-0251-4ede-9737-747e862fb4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:42.067297 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b5967-0251-4ede-9737-747e862fb4fd admin admin] PATH_INFO: `/v3/projects/ecc6657d44234486921e26e0519c08df/users/21739917e38a48469cad382fbdfdfaae/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:42.084401 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b5967-0251-4ede-9737-747e862fb4fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecc6657d44234486921e26e0519c08df, user_id=21739917e38a48469cad382fbdfdfaae, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:42.087242 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b5967-0251-4ede-9737-747e862fb4fd admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:42.165738 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1084/2188] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:42 2022] PUT /identity/v3/projects/ecc6657d44234486921e26e0519c08df/users/21739917e38a48469cad382fbdfdfaae/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:42.170220 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61438398-7ebf-4169-ad80-b3af97b1c462 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:42.185952 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61438398-7ebf-4169-ad80-b3af97b1c462 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:42.244248 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76b29c-113b-4dfb-a3ee-5599a912201f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:42.244998 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76b29c-113b-4dfb-a3ee-5599a912201f admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:42.246003 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76b29c-113b-4dfb-a3ee-5599a912201f admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:42.246373 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76b29c-113b-4dfb-a3ee-5599a912201f admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:42.247156 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76b29c-113b-4dfb-a3ee-5599a912201f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:42.249143 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76b29c-113b-4dfb-a3ee-5599a912201f admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:42.259964 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1105/2189] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:42 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:42.264018 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b5967-0251-4ede-9737-747e862fb4fd admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:42.282124 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b5967-0251-4ede-9737-747e862fb4fd admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:42.287117 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f70835-4d07-4916-be81-a7e50065166a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:42.287902 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f70835-4d07-4916-be81-a7e50065166a admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:42.288175 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f70835-4d07-4916-be81-a7e50065166a admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:42.288445 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f70835-4d07-4916-be81-a7e50065166a admin admin] PATH_INFO: `/v3/projects/ecc6657d44234486921e26e0519c08df/users/21739917e38a48469cad382fbdfdfaae/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:42.290329 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f70835-4d07-4916-be81-a7e50065166a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecc6657d44234486921e26e0519c08df, user_id=21739917e38a48469cad382fbdfdfaae, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:42.293482 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f70835-4d07-4916-be81-a7e50065166a admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:42.304091 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1085/2190] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:42 2022] PUT /identity/v3/projects/ecc6657d44234486921e26e0519c08df/users/21739917e38a48469cad382fbdfdfaae/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:06:47.692380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9fd2f-f270-48f9-be05-d0ed6819b677 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:47.692380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9fd2f-f270-48f9-be05-d0ed6819b677 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:47.692380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9fd2f-f270-48f9-be05-d0ed6819b677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:47.723117 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-35b9fd2f-f270-48f9-be05-d0ed6819b677 None None] MFA Rules not processed for user `21739917e38a48469cad382fbdfdfaae`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:47.724503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-35b9fd2f-f270-48f9-be05-d0ed6819b677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:47.729839 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:47.824742 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1106/2191] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:06:47 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:47.838197 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f70835-4d07-4916-be81-a7e50065166a admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:47.839899 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-96f70835-4d07-4916-be81-a7e50065166a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:47.906697 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f70835-4d07-4916-be81-a7e50065166a admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:47.910553 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c295b-3e6a-40d1-a0bb-cc7876a10f12 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2e72c8e7b444e978bf99c6404febcd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:47.911170 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c295b-3e6a-40d1-a0bb-cc7876a10f12 service glance] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:47.911446 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c295b-3e6a-40d1-a0bb-cc7876a10f12 service glance] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:47.911714 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c295b-3e6a-40d1-a0bb-cc7876a10f12 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:47.939082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:47.972421 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c295b-3e6a-40d1-a0bb-cc7876a10f12 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:47.973914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c295b-3e6a-40d1-a0bb-cc7876a10f12 service glance] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:47.986773 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1086/2192] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:48.271237 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef4247-599c-43cb-9322-fc3c31145004 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:48.271548 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef4247-599c-43cb-9322-fc3c31145004 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:48.271825 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef4247-599c-43cb-9322-fc3c31145004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:48.291611 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-f2ef4247-599c-43cb-9322-fc3c31145004 None None] MFA Rules not processed for user `7796ebff70054de090e02c58bf7000f4`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:48.293724 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-f2ef4247-599c-43cb-9322-fc3c31145004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:48.357983 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1107/2193] 10.222.0.16 () {58 vars in 1009 bytes} [Wed Aug 31 11:06:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:48.407691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c295b-3e6a-40d1-a0bb-cc7876a10f12 service glance] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:48.409536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-ae9c295b-3e6a-40d1-a0bb-cc7876a10f12 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:48.486831 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c295b-3e6a-40d1-a0bb-cc7876a10f12 service glance] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:48.491977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9007170-8abd-44fe-9960-b8464fc91e52 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8abc7c439a04622896b34ed0a7b8492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:48.492590 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9007170-8abd-44fe-9960-b8464fc91e52 service swift] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:48.492872 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9007170-8abd-44fe-9960-b8464fc91e52 service swift] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:48.493143 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9007170-8abd-44fe-9960-b8464fc91e52 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:48.502965 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9007170-8abd-44fe-9960-b8464fc91e52 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:48.504576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9007170-8abd-44fe-9960-b8464fc91e52 service swift] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:48.515826 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1087/2194] 10.222.0.16 () {60 vars in 1384 bytes} [Wed Aug 31 11:06:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 11:06:52.094674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84329ebb-3665-441c-b2d7-ee4129ca6643 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:52.095270 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84329ebb-3665-441c-b2d7-ee4129ca6643 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:52.095270 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84329ebb-3665-441c-b2d7-ee4129ca6643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:52.129196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-84329ebb-3665-441c-b2d7-ee4129ca6643 None None] MFA Rules not processed for user `927bd33801e34715a78b58b268e28c4b`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:52.131397 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-84329ebb-3665-441c-b2d7-ee4129ca6643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:52.207203 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1108/2195] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:06:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:58.511958 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=48 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:59.689678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb128f-f9b3-47df-9794-a359b623ccb3 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:59.690310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb128f-f9b3-47df-9794-a359b623ccb3 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:59.690310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb128f-f9b3-47df-9794-a359b623ccb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:59.711290 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-49bb128f-f9b3-47df-9794-a359b623ccb3 None None] MFA Rules not processed for user `82e5851d5c6a4a4c9168a64eb922a4d4`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:59.712546 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-49bb128f-f9b3-47df-9794-a359b623ccb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:59.720278 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1088/2196] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:06:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:07:02.203269 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=77 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:07:09.705351 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:07:13.143552 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc1a26-273e-4bfb-a932-31ff35c48ab0 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:13.144044 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc1a26-273e-4bfb-a932-31ff35c48ab0 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:13.144143 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc1a26-273e-4bfb-a932-31ff35c48ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:13.163560 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-d6cc1a26-273e-4bfb-a932-31ff35c48ab0 None None] MFA Rules not processed for user `bccf1c23c68a4987882eedad6ec5c106`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:07:13.164780 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-d6cc1a26-273e-4bfb-a932-31ff35c48ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:13.219530 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1109/2197] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:07:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:07:14.673832 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1f33e-8474-4f0b-b35d-9221d255af50 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:14.674337 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1f33e-8474-4f0b-b35d-9221d255af50 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:14.674337 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1f33e-8474-4f0b-b35d-9221d255af50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:14.696355 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-86c1f33e-8474-4f0b-b35d-9221d255af50 None None] MFA Rules not processed for user `7796ebff70054de090e02c58bf7000f4`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:07:14.697651 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-86c1f33e-8474-4f0b-b35d-9221d255af50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:14.710781 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1089/2198] 10.222.0.16 () {58 vars in 1009 bytes} [Wed Aug 31 11:07:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:07:14.721785 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc1a26-273e-4bfb-a932-31ff35c48ab0 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:14.736125 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc1a26-273e-4bfb-a932-31ff35c48ab0 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:14.745279 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4f5be-7d54-4de1-a95b-86a5f128d432 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8abc7c439a04622896b34ed0a7b8492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:14.745747 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4f5be-7d54-4de1-a95b-86a5f128d432 service swift] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:14.746102 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4f5be-7d54-4de1-a95b-86a5f128d432 service swift] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:14.746515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4f5be-7d54-4de1-a95b-86a5f128d432 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:14.757934 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e4f5be-7d54-4de1-a95b-86a5f128d432 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:14.761447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e4f5be-7d54-4de1-a95b-86a5f128d432 service swift] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:14.773365 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1110/2199] 10.222.0.16 () {60 vars in 1384 bytes} [Wed Aug 31 11:07:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:07:24.689064 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:07:24.768613 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=31 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:08:07.206932 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1f33e-8474-4f0b-b35d-9221d255af50 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:07.264241 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1f33e-8474-4f0b-b35d-9221d255af50 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:07.268563 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59295631-5d13-4a3f-929c-53073a463e62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:07.269182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59295631-5d13-4a3f-929c-53073a463e62 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:07.269607 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59295631-5d13-4a3f-929c-53073a463e62 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:07.269884 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59295631-5d13-4a3f-929c-53073a463e62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:07.280206 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59295631-5d13-4a3f-929c-53073a463e62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:07.282014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59295631-5d13-4a3f-929c-53073a463e62 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:07.293843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1090/2200] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:08:07 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:08:17.288166 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=26 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:08:30.702983 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4f5be-7d54-4de1-a95b-86a5f128d432 service swift] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:30.704434 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-01e4f5be-7d54-4de1-a95b-86a5f128d432 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:30.762186 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4f5be-7d54-4de1-a95b-86a5f128d432 service swift] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:30.765828 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7821ac-87b8-42eb-8d84-89833c967154 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:30.766456 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7821ac-87b8-42eb-8d84-89833c967154 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:30.766737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7821ac-87b8-42eb-8d84-89833c967154 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:30.767008 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7821ac-87b8-42eb-8d84-89833c967154 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:30.768834 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-7a7821ac-87b8-42eb-8d84-89833c967154 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:30.830082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7821ac-87b8-42eb-8d84-89833c967154 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:30.831610 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7821ac-87b8-42eb-8d84-89833c967154 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:30.841867 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1111/2201] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:08:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:08:31.597693 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59295631-5d13-4a3f-929c-53073a463e62 service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:31.599200 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-59295631-5d13-4a3f-929c-53073a463e62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:31.612066 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59295631-5d13-4a3f-929c-53073a463e62 service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:31.615572 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e7756-dd34-432b-807d-d7b845294091 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:31.616161 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e7756-dd34-432b-807d-d7b845294091 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:31.616432 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e7756-dd34-432b-807d-d7b845294091 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:31.616698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e7756-dd34-432b-807d-d7b845294091 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:31.618381 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-ea6e7756-dd34-432b-807d-d7b845294091 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:31.676057 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e7756-dd34-432b-807d-d7b845294091 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:31.677470 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e7756-dd34-432b-807d-d7b845294091 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:31.686361 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1091/2202] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:08:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:08:40.836426 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=48 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:08:41.682980 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=28 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:09:17.051898 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7821ac-87b8-42eb-8d84-89833c967154 service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:09:17.053372 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-7a7821ac-87b8-42eb-8d84-89833c967154 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:09:17.070114 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7821ac-87b8-42eb-8d84-89833c967154 service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:09:17.074701 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395480c-859e-4493-b8f4-0fd2e3d6be8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:09:17.075315 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395480c-859e-4493-b8f4-0fd2e3d6be8b service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:09:17.075589 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395480c-859e-4493-b8f4-0fd2e3d6be8b service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:09:17.075857 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395480c-859e-4493-b8f4-0fd2e3d6be8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:09:17.077734 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a395480c-859e-4493-b8f4-0fd2e3d6be8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:09:17.088026 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395480c-859e-4493-b8f4-0fd2e3d6be8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:09:17.089488 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395480c-859e-4493-b8f4-0fd2e3d6be8b service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:09:17.099292 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1112/2203] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:09:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:09:20.526546 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc7832-a46c-45de-97a6-36b4fb3a644f None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:09:20.527088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc7832-a46c-45de-97a6-36b4fb3a644f None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:09:20.527088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc7832-a46c-45de-97a6-36b4fb3a644f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:09:20.549361 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-09bc7832-a46c-45de-97a6-36b4fb3a644f None None] MFA Rules not processed for user `d5d0010f584249b4a22fd31634c9fd8b`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:09:20.550683 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-09bc7832-a46c-45de-97a6-36b4fb3a644f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:09:20.561500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1092/2204] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:09:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:09:20.602257 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395480c-859e-4493-b8f4-0fd2e3d6be8b service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:09:20.612197 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395480c-859e-4493-b8f4-0fd2e3d6be8b service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:09:20.616092 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fe9db-784e-4201-89ca-50a05f517725 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2e72c8e7b444e978bf99c6404febcd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:09:20.616694 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fe9db-784e-4201-89ca-50a05f517725 service glance] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:09:20.617099 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fe9db-784e-4201-89ca-50a05f517725 service glance] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:09:20.617214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fe9db-784e-4201-89ca-50a05f517725 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:09:20.623932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-c32fe9db-784e-4201-89ca-50a05f517725 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:09:20.702106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32fe9db-784e-4201-89ca-50a05f517725 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:09:20.703598 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32fe9db-784e-4201-89ca-50a05f517725 service glance] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:09:20.715587 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1113/2205] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:09:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:09:21.003862 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc7832-a46c-45de-97a6-36b4fb3a644f None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:09:21.014831 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc7832-a46c-45de-97a6-36b4fb3a644f None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:09:21.018506 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf050fb9-81c4-4480-8469-be646b282ec1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:09:21.019127 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf050fb9-81c4-4480-8469-be646b282ec1 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:09:21.019402 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf050fb9-81c4-4480-8469-be646b282ec1 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:09:21.019666 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf050fb9-81c4-4480-8469-be646b282ec1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:09:21.021406 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-cf050fb9-81c4-4480-8469-be646b282ec1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:09:21.032253 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf050fb9-81c4-4480-8469-be646b282ec1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:09:21.033670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf050fb9-81c4-4480-8469-be646b282ec1 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:09:21.042136 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1093/2206] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:09:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:09:21.452021 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fe9db-784e-4201-89ca-50a05f517725 service glance] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:09:21.453626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-c32fe9db-784e-4201-89ca-50a05f517725 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:09:21.461834 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf050fb9-81c4-4480-8469-be646b282ec1 service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:09:21.472288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf050fb9-81c4-4480-8469-be646b282ec1 service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:09:21.476548 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c60a1a-8a83-45b2-8e94-f3a4f325ceb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:09:21.477147 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c60a1a-8a83-45b2-8e94-f3a4f325ceb6 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:09:21.477423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c60a1a-8a83-45b2-8e94-f3a4f325ceb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:09:21.480369 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c60a1a-8a83-45b2-8e94-f3a4f325ceb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:09:21.480491 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-f4c60a1a-8a83-45b2-8e94-f3a4f325ceb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:09:21.495297 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c60a1a-8a83-45b2-8e94-f3a4f325ceb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:09:21.497136 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c60a1a-8a83-45b2-8e94-f3a4f325ceb6 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:09:21.508983 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1094/2207] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:09:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:09:21.537548 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fe9db-784e-4201-89ca-50a05f517725 service glance] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:09:21.542898 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9823b91-c8cb-4510-b4d8-ee9d60473b4d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3db46a4b4e74cc0bb2a7bd6f0e99105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:09:21.543512 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9823b91-c8cb-4510-b4d8-ee9d60473b4d service placement] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:09:21.543790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9823b91-c8cb-4510-b4d8-ee9d60473b4d service placement] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:09:21.544071 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9823b91-c8cb-4510-b4d8-ee9d60473b4d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:09:21.545906 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a9823b91-c8cb-4510-b4d8-ee9d60473b4d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:09:21.559897 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9823b91-c8cb-4510-b4d8-ee9d60473b4d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:09:21.562195 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9823b91-c8cb-4510-b4d8-ee9d60473b4d service placement] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:09:21.573341 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1114/2208] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:09:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:09:25.569264 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c60a1a-8a83-45b2-8e94-f3a4f325ceb6 service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:09:25.576132 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-f4c60a1a-8a83-45b2-8e94-f3a4f325ceb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:09:25.585765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c60a1a-8a83-45b2-8e94-f3a4f325ceb6 service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:09:25.591544 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567d6af-aa36-4050-aefa-fc218145103c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:09:25.592155 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567d6af-aa36-4050-aefa-fc218145103c service nova] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:09:25.592435 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567d6af-aa36-4050-aefa-fc218145103c service nova] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:09:25.592709 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567d6af-aa36-4050-aefa-fc218145103c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:09:25.594674 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-1567d6af-aa36-4050-aefa-fc218145103c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:09:25.608904 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567d6af-aa36-4050-aefa-fc218145103c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:09:25.610510 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567d6af-aa36-4050-aefa-fc218145103c service nova] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:09:25.623152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1095/2209] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:09:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:09:30.533756 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9823b91-c8cb-4510-b4d8-ee9d60473b4d service placement] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:09:30.543307 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9823b91-c8cb-4510-b4d8-ee9d60473b4d service placement] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:09:30.546916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aecfe62-33e1-474a-a420-100ac6420db1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:09:30.547512 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecfe62-33e1-474a-a420-100ac6420db1 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:09:30.547786 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecfe62-33e1-474a-a420-100ac6420db1 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:09:30.548051 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecfe62-33e1-474a-a420-100ac6420db1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:09:30.549746 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-7aecfe62-33e1-474a-a420-100ac6420db1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:09:30.562554 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aecfe62-33e1-474a-a420-100ac6420db1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:09:30.564020 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aecfe62-33e1-474a-a420-100ac6420db1 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:09:30.574754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1115/2210] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:09:30 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:09:35.617155 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=21 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:09:35.781675 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567d6af-aa36-4050-aefa-fc218145103c service nova] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:09:35.795272 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567d6af-aa36-4050-aefa-fc218145103c service nova] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:09:35.801189 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd11c21-85c2-4809-a9e0-31a077dd6f13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:09:35.801798 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd11c21-85c2-4809-a9e0-31a077dd6f13 service nova] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:09:35.802073 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd11c21-85c2-4809-a9e0-31a077dd6f13 service nova] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:09:35.802344 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd11c21-85c2-4809-a9e0-31a077dd6f13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:09:35.804294 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-4dd11c21-85c2-4809-a9e0-31a077dd6f13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:09:35.818382 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd11c21-85c2-4809-a9e0-31a077dd6f13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:09:35.820110 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd11c21-85c2-4809-a9e0-31a077dd6f13 service nova] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:09:35.831673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1096/2211] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:09:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:09:40.569546 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=78 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:09:41.597241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aecfe62-33e1-474a-a420-100ac6420db1 service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:09:41.610288 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aecfe62-33e1-474a-a420-100ac6420db1 service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:09:41.615782 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977862af-f13a-4a1a-bc3d-c2a93170fbf9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:09:41.616380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977862af-f13a-4a1a-bc3d-c2a93170fbf9 service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:09:41.616657 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977862af-f13a-4a1a-bc3d-c2a93170fbf9 service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:09:41.616941 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977862af-f13a-4a1a-bc3d-c2a93170fbf9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:09:41.618935 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-977862af-f13a-4a1a-bc3d-c2a93170fbf9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:09:41.630974 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977862af-f13a-4a1a-bc3d-c2a93170fbf9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:09:41.632557 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977862af-f13a-4a1a-bc3d-c2a93170fbf9 service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:09:41.643905 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1116/2212] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:09:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:09:45.827384 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=6 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:09:51.639432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=6 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:09:52.234088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd11c21-85c2-4809-a9e0-31a077dd6f13 service nova] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:09:52.235538 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-4dd11c21-85c2-4809-a9e0-31a077dd6f13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:09:52.249264 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd11c21-85c2-4809-a9e0-31a077dd6f13 service nova] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:09:52.254775 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d3d355-5e2e-4e4c-9006-0bd406bdf04a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:09:52.255362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3d355-5e2e-4e4c-9006-0bd406bdf04a service nova] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:09:52.255636 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3d355-5e2e-4e4c-9006-0bd406bdf04a service nova] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:09:52.255903 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3d355-5e2e-4e4c-9006-0bd406bdf04a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:09:52.257557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-52d3d355-5e2e-4e4c-9006-0bd406bdf04a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:09:52.268067 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d3d355-5e2e-4e4c-9006-0bd406bdf04a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:09:52.270314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d3d355-5e2e-4e4c-9006-0bd406bdf04a service nova] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:09:52.279100 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1097/2213] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:09:52 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:10:02.274422 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=6 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:10:59.195126 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977862af-f13a-4a1a-bc3d-c2a93170fbf9 service nova] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:59.205550 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977862af-f13a-4a1a-bc3d-c2a93170fbf9 service nova] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:59.209203 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750dbff-f295-4023-85ac-8d2f029c5d43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:59.209803 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750dbff-f295-4023-85ac-8d2f029c5d43 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:59.210078 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750dbff-f295-4023-85ac-8d2f029c5d43 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:59.210348 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750dbff-f295-4023-85ac-8d2f029c5d43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:59.212184 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-c750dbff-f295-4023-85ac-8d2f029c5d43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:10:59.273633 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750dbff-f295-4023-85ac-8d2f029c5d43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:59.275112 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750dbff-f295-4023-85ac-8d2f029c5d43 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:59.284691 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1117/2214] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:10:59 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:11:09.279894 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=28 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:31.895397 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d3d355-5e2e-4e4c-9006-0bd406bdf04a service nova] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:31.960974 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d3d355-5e2e-4e4c-9006-0bd406bdf04a service nova] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:31.966228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42218c36-c249-4f3b-9ff5-dc8b228f58c1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3db46a4b4e74cc0bb2a7bd6f0e99105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:31.966861 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42218c36-c249-4f3b-9ff5-dc8b228f58c1 service placement] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:31.967186 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42218c36-c249-4f3b-9ff5-dc8b228f58c1 service placement] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:31.967556 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42218c36-c249-4f3b-9ff5-dc8b228f58c1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:31.969326 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-42218c36-c249-4f3b-9ff5-dc8b228f58c1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:32.003839 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42218c36-c249-4f3b-9ff5-dc8b228f58c1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:32.005293 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42218c36-c249-4f3b-9ff5-dc8b228f58c1 service placement] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:32.016321 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1098/2215] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:11:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:11:42.012029 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=36 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:49.284441 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750dbff-f295-4023-85ac-8d2f029c5d43 service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:49.339096 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750dbff-f295-4023-85ac-8d2f029c5d43 service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:49.345749 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae16fbd-4d4f-493b-b891-e175d7280467 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:49.346348 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae16fbd-4d4f-493b-b891-e175d7280467 service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:49.346661 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae16fbd-4d4f-493b-b891-e175d7280467 service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:49.346928 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae16fbd-4d4f-493b-b891-e175d7280467 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:49.354927 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae16fbd-4d4f-493b-b891-e175d7280467 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:49.356504 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae16fbd-4d4f-493b-b891-e175d7280467 service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:49.366539 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1118/2216] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:11:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:11:59.362574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=24 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:31.146512 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42218c36-c249-4f3b-9ff5-dc8b228f58c1 service placement] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:31.165307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42218c36-c249-4f3b-9ff5-dc8b228f58c1 service placement] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:31.168986 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2247d3-6f77-4a8b-8a91-6c25315e4cd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:31.169584 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2247d3-6f77-4a8b-8a91-6c25315e4cd4 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:31.169861 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2247d3-6f77-4a8b-8a91-6c25315e4cd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:31.170130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2247d3-6f77-4a8b-8a91-6c25315e4cd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:31.180081 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2247d3-6f77-4a8b-8a91-6c25315e4cd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:31.181533 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2247d3-6f77-4a8b-8a91-6c25315e4cd4 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:31.209237 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1099/2217] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:13:32.451755 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae16fbd-4d4f-493b-b891-e175d7280467 service nova] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:32.467474 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae16fbd-4d4f-493b-b891-e175d7280467 service nova] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:32.475441 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194293a-8b0a-4c89-9512-361e30ecc681 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:32.476066 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194293a-8b0a-4c89-9512-361e30ecc681 service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:32.476346 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194293a-8b0a-4c89-9512-361e30ecc681 service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:32.476620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194293a-8b0a-4c89-9512-361e30ecc681 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:32.486998 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194293a-8b0a-4c89-9512-361e30ecc681 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:32.492540 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194293a-8b0a-4c89-9512-361e30ecc681 service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:32.520718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1119/2218] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:13:41.206563 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=17 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:42.517906 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=13 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:20.436176 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2247d3-6f77-4a8b-8a91-6c25315e4cd4 service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:20.457494 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2247d3-6f77-4a8b-8a91-6c25315e4cd4 service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:20.461226 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dfec1f-d22c-4814-8969-ce2a1d519995 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:20.461829 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfec1f-d22c-4814-8969-ce2a1d519995 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:20.462101 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfec1f-d22c-4814-8969-ce2a1d519995 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:20.462367 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfec1f-d22c-4814-8969-ce2a1d519995 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:20.474222 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dfec1f-d22c-4814-8969-ce2a1d519995 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:20.475726 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dfec1f-d22c-4814-8969-ce2a1d519995 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:20.488215 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1100/2219] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:14:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:14:25.148332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dabe54-7aa2-4d50-a56f-3d53f3c74685 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:25.148947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dabe54-7aa2-4d50-a56f-3d53f3c74685 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:25.148947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dabe54-7aa2-4d50-a56f-3d53f3c74685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:25.190114 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-f3dabe54-7aa2-4d50-a56f-3d53f3c74685 None None] MFA Rules not processed for user `5d1ce9150ba543b1b06f8edb5a545af0`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:14:25.191423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-f3dabe54-7aa2-4d50-a56f-3d53f3c74685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:25.198328 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1120/2220] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:14:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:14:25.237372 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dfec1f-d22c-4814-8969-ce2a1d519995 service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:25.246896 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dfec1f-d22c-4814-8969-ce2a1d519995 service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:25.251292 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e1b90-45e4-4827-b54e-c4f15f2b18a4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2e72c8e7b444e978bf99c6404febcd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:25.251896 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e1b90-45e4-4827-b54e-c4f15f2b18a4 service glance] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:25.252172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e1b90-45e4-4827-b54e-c4f15f2b18a4 service glance] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:25.252439 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e1b90-45e4-4827-b54e-c4f15f2b18a4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:25.265717 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e1b90-45e4-4827-b54e-c4f15f2b18a4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:25.267182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e1b90-45e4-4827-b54e-c4f15f2b18a4 service glance] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:25.278614 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1101/2221] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:14:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:14:25.571278 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dabe54-7aa2-4d50-a56f-3d53f3c74685 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:25.581629 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dabe54-7aa2-4d50-a56f-3d53f3c74685 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:25.585247 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a021f8c7-8f81-41fe-98ad-942db8f0ed9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:25.585836 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021f8c7-8f81-41fe-98ad-942db8f0ed9b service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:25.586109 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021f8c7-8f81-41fe-98ad-942db8f0ed9b service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:25.586375 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021f8c7-8f81-41fe-98ad-942db8f0ed9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:25.593657 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a021f8c7-8f81-41fe-98ad-942db8f0ed9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:25.595125 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a021f8c7-8f81-41fe-98ad-942db8f0ed9b service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:25.603443 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1121/2222] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:14:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:14:26.009035 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e1b90-45e4-4827-b54e-c4f15f2b18a4 service glance] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:26.021312 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e1b90-45e4-4827-b54e-c4f15f2b18a4 service glance] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:26.026310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824fb0a-1dc2-4d71-b732-5b464e9a803f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3db46a4b4e74cc0bb2a7bd6f0e99105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:26.026946 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824fb0a-1dc2-4d71-b732-5b464e9a803f service placement] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:26.027217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824fb0a-1dc2-4d71-b732-5b464e9a803f service placement] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:26.027483 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824fb0a-1dc2-4d71-b732-5b464e9a803f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:26.037120 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b824fb0a-1dc2-4d71-b732-5b464e9a803f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:26.038736 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b824fb0a-1dc2-4d71-b732-5b464e9a803f service placement] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:26.050118 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1102/2223] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:14:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:14:29.240096 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a021f8c7-8f81-41fe-98ad-942db8f0ed9b service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:29.249363 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a021f8c7-8f81-41fe-98ad-942db8f0ed9b service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:29.259397 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6025c38-2e97-4c58-84b4-479b2600ec9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:29.259397 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6025c38-2e97-4c58-84b4-479b2600ec9e service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:29.259397 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6025c38-2e97-4c58-84b4-479b2600ec9e service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:29.259397 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6025c38-2e97-4c58-84b4-479b2600ec9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:29.262935 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6025c38-2e97-4c58-84b4-479b2600ec9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:29.264658 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6025c38-2e97-4c58-84b4-479b2600ec9e service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:29.277450 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1122/2224] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:14:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:14:29.501246 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824fb0a-1dc2-4d71-b732-5b464e9a803f service placement] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:29.512415 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824fb0a-1dc2-4d71-b732-5b464e9a803f service placement] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:29.516945 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acad325-493a-4a8e-acbf-a268629a7687 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:29.519097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acad325-493a-4a8e-acbf-a268629a7687 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:29.519097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acad325-493a-4a8e-acbf-a268629a7687 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:29.519097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acad325-493a-4a8e-acbf-a268629a7687 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:29.527741 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acad325-493a-4a8e-acbf-a268629a7687 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:29.529872 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acad325-493a-4a8e-acbf-a268629a7687 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:29.540598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1103/2225] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:14:29 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:14:30.586088 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6025c38-2e97-4c58-84b4-479b2600ec9e service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:30.600928 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6025c38-2e97-4c58-84b4-479b2600ec9e service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:30.607027 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325666f-0719-4f55-9b9e-5e6f767f3cd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:30.608286 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325666f-0719-4f55-9b9e-5e6f767f3cd4 service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:30.608741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325666f-0719-4f55-9b9e-5e6f767f3cd4 service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:30.609045 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325666f-0719-4f55-9b9e-5e6f767f3cd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:30.622307 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325666f-0719-4f55-9b9e-5e6f767f3cd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:30.624300 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325666f-0719-4f55-9b9e-5e6f767f3cd4 service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:30.638007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1123/2226] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:14:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:14:37.206910 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acad325-493a-4a8e-acbf-a268629a7687 service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:37.221119 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acad325-493a-4a8e-acbf-a268629a7687 service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:37.227531 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e0a836-776f-4a32-a5b8-816daa0ba3f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:37.228157 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e0a836-776f-4a32-a5b8-816daa0ba3f4 service nova] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:37.228455 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e0a836-776f-4a32-a5b8-816daa0ba3f4 service nova] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:37.228732 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e0a836-776f-4a32-a5b8-816daa0ba3f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:37.239077 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e0a836-776f-4a32-a5b8-816daa0ba3f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:37.240945 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e0a836-776f-4a32-a5b8-816daa0ba3f4 service nova] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:37.253404 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1104/2227] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:14:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:14:40.631430 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=29 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:47.248080 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=34 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:52.264109 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325666f-0719-4f55-9b9e-5e6f767f3cd4 service nova] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:52.273454 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325666f-0719-4f55-9b9e-5e6f767f3cd4 service nova] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:52.277059 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf7f54-5f46-4159-b8f2-8b5a26a128f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:52.277646 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf7f54-5f46-4159-b8f2-8b5a26a128f4 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:52.277926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf7f54-5f46-4159-b8f2-8b5a26a128f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:52.278241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf7f54-5f46-4159-b8f2-8b5a26a128f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:52.287423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf7f54-5f46-4159-b8f2-8b5a26a128f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:52.288851 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf7f54-5f46-4159-b8f2-8b5a26a128f4 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:52.316889 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1124/2228] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:14:52 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:15:02.314174 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=13 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:15:09.220780 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb63837-fe13-4baa-9da9-0385949bb2cd None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:15:09.221276 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb63837-fe13-4baa-9da9-0385949bb2cd None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:15:09.221390 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb63837-fe13-4baa-9da9-0385949bb2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:15:09.241913 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-5eb63837-fe13-4baa-9da9-0385949bb2cd None None] MFA Rules not processed for user `8ab477a1ab0d4e61b9649b62ee5295a0`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:15:09.243145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-5eb63837-fe13-4baa-9da9-0385949bb2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:15:09.250805 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1105/2229] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:15:09 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:15:09.583410 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf7f54-5f46-4159-b8f2-8b5a26a128f4 service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:15:09.593567 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf7f54-5f46-4159-b8f2-8b5a26a128f4 service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:15:09.597494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3cd1ec-5c9a-49af-bddc-197891fe566b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:15:09.598106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cd1ec-5c9a-49af-bddc-197891fe566b service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:15:09.598385 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cd1ec-5c9a-49af-bddc-197891fe566b service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:15:09.598686 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cd1ec-5c9a-49af-bddc-197891fe566b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:15:09.600480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-1d3cd1ec-5c9a-49af-bddc-197891fe566b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:15:09.626018 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3cd1ec-5c9a-49af-bddc-197891fe566b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:15:09.627545 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3cd1ec-5c9a-49af-bddc-197891fe566b service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:15:09.636375 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1125/2230] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:15:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:15:19.236806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:15:19.632449 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=11 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:16:32.864993 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb63837-fe13-4baa-9da9-0385949bb2cd None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:16:32.877355 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb63837-fe13-4baa-9da9-0385949bb2cd None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:16:32.882274 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582bd435-c03c-4977-b83b-0950a8e34d33 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3db46a4b4e74cc0bb2a7bd6f0e99105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:16:32.882915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582bd435-c03c-4977-b83b-0950a8e34d33 service placement] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:16:32.883213 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582bd435-c03c-4977-b83b-0950a8e34d33 service placement] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:16:32.883482 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582bd435-c03c-4977-b83b-0950a8e34d33 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:16:32.893162 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582bd435-c03c-4977-b83b-0950a8e34d33 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:16:32.894808 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582bd435-c03c-4977-b83b-0950a8e34d33 service placement] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:16:32.904990 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1106/2231] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:16:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:16:42.900496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=6 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:16:49.640612 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3cd1ec-5c9a-49af-bddc-197891fe566b service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:16:49.654089 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3cd1ec-5c9a-49af-bddc-197891fe566b service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:16:49.659624 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6bd04-a89c-4f92-b97d-c25897b4bbff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:16:49.660238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6bd04-a89c-4f92-b97d-c25897b4bbff service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:16:49.660518 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6bd04-a89c-4f92-b97d-c25897b4bbff service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:16:49.660792 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6bd04-a89c-4f92-b97d-c25897b4bbff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:16:49.662695 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-10d6bd04-a89c-4f92-b97d-c25897b4bbff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:16:49.743883 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6bd04-a89c-4f92-b97d-c25897b4bbff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:16:49.746355 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6bd04-a89c-4f92-b97d-c25897b4bbff service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:16:49.765289 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1126/2232] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:16:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:16:55.131327 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582bd435-c03c-4977-b83b-0950a8e34d33 service placement] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:16:55.143632 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582bd435-c03c-4977-b83b-0950a8e34d33 service placement] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:16:55.149634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffc3d2-a3e5-49c0-bb62-2d486b0d49a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:16:55.150226 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffc3d2-a3e5-49c0-bb62-2d486b0d49a5 service nova] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:16:55.150531 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffc3d2-a3e5-49c0-bb62-2d486b0d49a5 service nova] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:16:55.150806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffc3d2-a3e5-49c0-bb62-2d486b0d49a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:16:55.162845 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bffc3d2-a3e5-49c0-bb62-2d486b0d49a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:16:55.164429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bffc3d2-a3e5-49c0-bb62-2d486b0d49a5 service nova] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:16:55.192998 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1107/2233] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:16:55 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:16:59.762688 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=41 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:17:05.190272 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=13 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:18:30.061010 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6bd04-a89c-4f92-b97d-c25897b4bbff service nova] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:18:30.070870 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6bd04-a89c-4f92-b97d-c25897b4bbff service nova] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:18:30.074518 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab55b5b-e27d-4ff3-b4b2-74632989eae7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:18:30.075112 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab55b5b-e27d-4ff3-b4b2-74632989eae7 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:18:30.075385 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab55b5b-e27d-4ff3-b4b2-74632989eae7 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:18:30.075653 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab55b5b-e27d-4ff3-b4b2-74632989eae7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:18:30.085125 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab55b5b-e27d-4ff3-b4b2-74632989eae7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:18:30.086572 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab55b5b-e27d-4ff3-b4b2-74632989eae7 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:18:30.097626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1127/2234] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:18:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:18:31.874084 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffc3d2-a3e5-49c0-bb62-2d486b0d49a5 service nova] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:18:31.875816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-7bffc3d2-a3e5-49c0-bb62-2d486b0d49a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:18:31.943921 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffc3d2-a3e5-49c0-bb62-2d486b0d49a5 service nova] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:18:31.947562 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934883b3-ea4b-4b00-ae75-6229a71f55c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:18:31.948167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934883b3-ea4b-4b00-ae75-6229a71f55c8 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:18:31.948448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934883b3-ea4b-4b00-ae75-6229a71f55c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:18:31.948720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934883b3-ea4b-4b00-ae75-6229a71f55c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:18:31.950617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-934883b3-ea4b-4b00-ae75-6229a71f55c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:18:32.023413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934883b3-ea4b-4b00-ae75-6229a71f55c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:18:32.024858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934883b3-ea4b-4b00-ae75-6229a71f55c8 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:18:32.034099 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1108/2235] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:18:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:18:40.091491 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=6 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:18:42.029771 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=58 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:19:29.299405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab55b5b-e27d-4ff3-b4b2-74632989eae7 service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:19:29.301114 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-3ab55b5b-e27d-4ff3-b4b2-74632989eae7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:19:29.357527 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab55b5b-e27d-4ff3-b4b2-74632989eae7 service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:19:29.361950 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671087d1-1a06-4611-bff1-492c782fc521 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:19:29.362519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671087d1-1a06-4611-bff1-492c782fc521 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:19:29.362803 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671087d1-1a06-4611-bff1-492c782fc521 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:19:29.363074 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671087d1-1a06-4611-bff1-492c782fc521 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:19:29.427743 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671087d1-1a06-4611-bff1-492c782fc521 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:19:29.429493 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671087d1-1a06-4611-bff1-492c782fc521 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:19:29.448220 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1128/2236] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:19:29 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:19:34.502303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934883b3-ea4b-4b00-ae75-6229a71f55c8 service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:19:34.503953 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-934883b3-ea4b-4b00-ae75-6229a71f55c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:19:34.561138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934883b3-ea4b-4b00-ae75-6229a71f55c8 service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:19:34.566254 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502c82f-ff35-4746-a492-a03558ce2f65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:19:34.566904 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502c82f-ff35-4746-a492-a03558ce2f65 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:19:34.567466 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502c82f-ff35-4746-a492-a03558ce2f65 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:19:34.567867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502c82f-ff35-4746-a492-a03558ce2f65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:19:34.569776 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-f502c82f-ff35-4746-a492-a03558ce2f65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:19:34.580765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502c82f-ff35-4746-a492-a03558ce2f65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:19:34.582186 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502c82f-ff35-4746-a492-a03558ce2f65 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:19:34.590163 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1109/2237] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:19:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:19:39.445619 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=56 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:19:44.586689 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=26 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:19:53.303566 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671087d1-1a06-4611-bff1-492c782fc521 service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:19:53.312378 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671087d1-1a06-4611-bff1-492c782fc521 service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:19:53.316009 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d0d4e-4404-4b34-906b-7cc3cb84aed2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:19:53.316602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d0d4e-4404-4b34-906b-7cc3cb84aed2 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:19:53.316873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d0d4e-4404-4b34-906b-7cc3cb84aed2 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:19:53.317136 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d0d4e-4404-4b34-906b-7cc3cb84aed2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:19:53.319008 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-1a1d0d4e-4404-4b34-906b-7cc3cb84aed2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:19:53.392739 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d0d4e-4404-4b34-906b-7cc3cb84aed2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:19:53.394106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d0d4e-4404-4b34-906b-7cc3cb84aed2 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:19:53.403114 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1129/2238] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:19:53 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:20:03.398916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=33 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:20:16.534857 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502c82f-ff35-4746-a492-a03558ce2f65 service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:20:16.543597 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502c82f-ff35-4746-a492-a03558ce2f65 service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:20:16.547639 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63b2c9-92c0-4f2d-86d8-fb2ac1876a0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:20:16.548265 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63b2c9-92c0-4f2d-86d8-fb2ac1876a0a service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:20:16.548544 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63b2c9-92c0-4f2d-86d8-fb2ac1876a0a service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:20:16.548814 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63b2c9-92c0-4f2d-86d8-fb2ac1876a0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:20:16.550741 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-1d63b2c9-92c0-4f2d-86d8-fb2ac1876a0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:20:16.644728 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63b2c9-92c0-4f2d-86d8-fb2ac1876a0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:20:16.646196 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63b2c9-92c0-4f2d-86d8-fb2ac1876a0a service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:20:16.657785 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1110/2239] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:20:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:20:17.847664 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d0d4e-4404-4b34-906b-7cc3cb84aed2 service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:20:17.849261 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-1a1d0d4e-4404-4b34-906b-7cc3cb84aed2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:20:17.864475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d0d4e-4404-4b34-906b-7cc3cb84aed2 service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:20:17.868156 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ef02a-9ade-4942-8507-a339adcd1fb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:20:17.868752 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ef02a-9ade-4942-8507-a339adcd1fb8 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:20:17.869029 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ef02a-9ade-4942-8507-a339adcd1fb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:20:17.869298 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ef02a-9ade-4942-8507-a339adcd1fb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:20:17.871129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-e57ef02a-9ade-4942-8507-a339adcd1fb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:20:17.884748 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ef02a-9ade-4942-8507-a339adcd1fb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:20:17.886111 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ef02a-9ade-4942-8507-a339adcd1fb8 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:20:17.898521 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1130/2240] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:20:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:20:26.652552 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=39 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:20:27.892734 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=6 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:21:38.857051 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63b2c9-92c0-4f2d-86d8-fb2ac1876a0a service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:21:38.858641 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-1d63b2c9-92c0-4f2d-86d8-fb2ac1876a0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:21:38.984359 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63b2c9-92c0-4f2d-86d8-fb2ac1876a0a service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:21:38.989499 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f28edf-49f5-4294-af62-6e58a2226c55 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3db46a4b4e74cc0bb2a7bd6f0e99105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:21:38.990107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f28edf-49f5-4294-af62-6e58a2226c55 service placement] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:21:38.990380 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f28edf-49f5-4294-af62-6e58a2226c55 service placement] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:21:38.990678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f28edf-49f5-4294-af62-6e58a2226c55 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:21:38.992566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-79f28edf-49f5-4294-af62-6e58a2226c55 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:21:39.004429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f28edf-49f5-4294-af62-6e58a2226c55 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:21:39.006060 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f28edf-49f5-4294-af62-6e58a2226c55 service placement] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:21:39.016852 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1111/2241] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:21:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:21:49.012626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=52 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:21:50.721942 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ef02a-9ade-4942-8507-a339adcd1fb8 service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:21:50.741827 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ef02a-9ade-4942-8507-a339adcd1fb8 service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:21:50.746052 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42a02b-4957-4d11-9168-7f028b743b3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:21:50.746759 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42a02b-4957-4d11-9168-7f028b743b3e service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:21:50.747033 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42a02b-4957-4d11-9168-7f028b743b3e service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:21:50.747300 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42a02b-4957-4d11-9168-7f028b743b3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:21:50.770662 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42a02b-4957-4d11-9168-7f028b743b3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:21:50.772086 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42a02b-4957-4d11-9168-7f028b743b3e service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:21:50.781259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1131/2242] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:21:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:21:55.099646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f28edf-49f5-4294-af62-6e58a2226c55 service placement] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:21:55.123374 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f28edf-49f5-4294-af62-6e58a2226c55 service placement] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:21:55.127035 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0462d8e7-7cdc-4ab4-adb9-99a37dd94780 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:21:55.127631 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462d8e7-7cdc-4ab4-adb9-99a37dd94780 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:21:55.127906 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462d8e7-7cdc-4ab4-adb9-99a37dd94780 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:21:55.128175 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462d8e7-7cdc-4ab4-adb9-99a37dd94780 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:21:55.130051 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-0462d8e7-7cdc-4ab4-adb9-99a37dd94780 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:21:55.206928 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0462d8e7-7cdc-4ab4-adb9-99a37dd94780 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:21:55.208362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0462d8e7-7cdc-4ab4-adb9-99a37dd94780 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:21:55.218262 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1112/2243] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:21:55 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:22:00.248463 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42a02b-4957-4d11-9168-7f028b743b3e service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:22:00.251903 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-0e42a02b-4957-4d11-9168-7f028b743b3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:22:00.284538 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42a02b-4957-4d11-9168-7f028b743b3e service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:22:00.290675 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc81b6-140a-4055-a0a5-6fb8cb3917ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:22:00.291295 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc81b6-140a-4055-a0a5-6fb8cb3917ef service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:22:00.291570 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc81b6-140a-4055-a0a5-6fb8cb3917ef service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:22:00.291838 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc81b6-140a-4055-a0a5-6fb8cb3917ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:22:00.293828 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-d5dc81b6-140a-4055-a0a5-6fb8cb3917ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:22:00.313410 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc81b6-140a-4055-a0a5-6fb8cb3917ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:22:00.313410 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc81b6-140a-4055-a0a5-6fb8cb3917ef service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:22:00.350889 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1132/2244] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:22:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:22:05.213229 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=38 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:22:10.333530 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=30 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:23:30.327922 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0462d8e7-7cdc-4ab4-adb9-99a37dd94780 service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:23:30.337116 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0462d8e7-7cdc-4ab4-adb9-99a37dd94780 service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:23:30.341061 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2066b-595f-497c-99fa-f08bbef05bdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:23:30.341672 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2066b-595f-497c-99fa-f08bbef05bdf service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:23:30.341950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2066b-595f-497c-99fa-f08bbef05bdf service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:23:30.343293 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2066b-595f-497c-99fa-f08bbef05bdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:23:30.344919 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-79b2066b-595f-497c-99fa-f08bbef05bdf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:23:30.406862 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b2066b-595f-497c-99fa-f08bbef05bdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:23:30.408318 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b2066b-595f-497c-99fa-f08bbef05bdf service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:23:30.417994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1113/2245] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:23:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:23:32.939951 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc81b6-140a-4055-a0a5-6fb8cb3917ef service nova] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:23:32.959074 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc81b6-140a-4055-a0a5-6fb8cb3917ef service nova] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:23:32.962682 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37410b4-13fe-4711-a491-ae4da908c558 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:23:32.963269 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37410b4-13fe-4711-a491-ae4da908c558 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:23:32.963541 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37410b4-13fe-4711-a491-ae4da908c558 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:23:32.963809 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37410b4-13fe-4711-a491-ae4da908c558 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:23:32.973669 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37410b4-13fe-4711-a491-ae4da908c558 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:23:32.975139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37410b4-13fe-4711-a491-ae4da908c558 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:23:33.003346 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1133/2246] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:23:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:23:40.413449 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=28 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:23:43.000638 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=17 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:24:30.158139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2066b-595f-497c-99fa-f08bbef05bdf service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:24:30.171318 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2066b-595f-497c-99fa-f08bbef05bdf service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:24:30.176881 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12cc87-f87f-45ec-bcee-40eb32c35826 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:24:30.177475 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12cc87-f87f-45ec-bcee-40eb32c35826 service nova] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:24:30.177760 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12cc87-f87f-45ec-bcee-40eb32c35826 service nova] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:24:30.178029 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12cc87-f87f-45ec-bcee-40eb32c35826 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:24:30.179896 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-1f12cc87-f87f-45ec-bcee-40eb32c35826 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:24:30.203920 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f12cc87-f87f-45ec-bcee-40eb32c35826 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:24:30.205495 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f12cc87-f87f-45ec-bcee-40eb32c35826 service nova] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:24:30.214471 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1114/2247] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:24:30 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:24:38.509956 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37410b4-13fe-4711-a491-ae4da908c558 service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:24:38.529280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37410b4-13fe-4711-a491-ae4da908c558 service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:24:38.533229 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb3da6-2719-4563-aa52-82c4d74644ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:24:38.533827 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb3da6-2719-4563-aa52-82c4d74644ca service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:24:38.534100 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb3da6-2719-4563-aa52-82c4d74644ca service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:24:38.534532 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb3da6-2719-4563-aa52-82c4d74644ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:24:38.542366 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb3da6-2719-4563-aa52-82c4d74644ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:24:38.543845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb3da6-2719-4563-aa52-82c4d74644ca service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:24:38.555999 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1134/2248] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:24:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:24:40.210361 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=11 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:24:48.552141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=10 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:25:17.067278 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12cc87-f87f-45ec-bcee-40eb32c35826 service nova] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:25:17.077847 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12cc87-f87f-45ec-bcee-40eb32c35826 service nova] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:25:17.081693 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7253f28-6cda-45e2-b7e9-77b7cc29e0d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:25:17.088078 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7253f28-6cda-45e2-b7e9-77b7cc29e0d9 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:25:17.088078 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7253f28-6cda-45e2-b7e9-77b7cc29e0d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:25:17.088078 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7253f28-6cda-45e2-b7e9-77b7cc29e0d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:25:17.094350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7253f28-6cda-45e2-b7e9-77b7cc29e0d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:25:17.095941 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7253f28-6cda-45e2-b7e9-77b7cc29e0d9 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:25:17.107827 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1115/2249] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:25:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:25:18.348047 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb3da6-2719-4563-aa52-82c4d74644ca service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:25:18.357340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb3da6-2719-4563-aa52-82c4d74644ca service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:25:18.361103 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7be2d5-14b5-489b-8b64-4c9c823e99a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:25:18.361629 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7be2d5-14b5-489b-8b64-4c9c823e99a8 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:25:18.361902 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7be2d5-14b5-489b-8b64-4c9c823e99a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:25:18.362167 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7be2d5-14b5-489b-8b64-4c9c823e99a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:25:18.372100 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7be2d5-14b5-489b-8b64-4c9c823e99a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:25:18.373519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7be2d5-14b5-489b-8b64-4c9c823e99a8 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:25:18.384205 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1135/2250] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:25:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:25:22.279255 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7253f28-6cda-45e2-b7e9-77b7cc29e0d9 service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:25:22.288615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7253f28-6cda-45e2-b7e9-77b7cc29e0d9 service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:25:22.292285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df60c01-efbb-453d-a152-62c86761e2c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:25:22.293034 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df60c01-efbb-453d-a152-62c86761e2c3 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:25:22.293372 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df60c01-efbb-453d-a152-62c86761e2c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:25:22.293676 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df60c01-efbb-453d-a152-62c86761e2c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:25:22.302192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df60c01-efbb-453d-a152-62c86761e2c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:25:22.303640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df60c01-efbb-453d-a152-62c86761e2c3 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:25:22.336560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1116/2251] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:25:22 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:25:28.379588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=6 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:25:32.333800 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=19 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:26:43.793775 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7be2d5-14b5-489b-8b64-4c9c823e99a8 service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:26:43.795387 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-6a7be2d5-14b5-489b-8b64-4c9c823e99a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:26:43.811853 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7be2d5-14b5-489b-8b64-4c9c823e99a8 service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:26:43.816867 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7c668-246d-4c9f-91b3-e4ef61b935b7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3db46a4b4e74cc0bb2a7bd6f0e99105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:26:43.817463 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7c668-246d-4c9f-91b3-e4ef61b935b7 service placement] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:26:43.817745 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7c668-246d-4c9f-91b3-e4ef61b935b7 service placement] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:26:43.818016 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7c668-246d-4c9f-91b3-e4ef61b935b7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:26:43.826576 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca7c668-246d-4c9f-91b3-e4ef61b935b7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:26:43.828157 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca7c668-246d-4c9f-91b3-e4ef61b935b7 service placement] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:26:43.838088 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1136/2252] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:26:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:26:50.353571 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df60c01-efbb-453d-a152-62c86761e2c3 service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:26:50.366334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df60c01-efbb-453d-a152-62c86761e2c3 service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:26:50.371621 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5983b80-c338-47a3-96d7-b92463563c4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:26:50.372209 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5983b80-c338-47a3-96d7-b92463563c4f service nova] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:26:50.372483 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5983b80-c338-47a3-96d7-b92463563c4f service nova] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:26:50.373455 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5983b80-c338-47a3-96d7-b92463563c4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:26:50.375408 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a5983b80-c338-47a3-96d7-b92463563c4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:26:50.454114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5983b80-c338-47a3-96d7-b92463563c4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:26:50.455730 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5983b80-c338-47a3-96d7-b92463563c4f service nova] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:26:50.474033 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1117/2253] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:26:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:26:53.834605 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=6 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:27:00.471516 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=41 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:27:22.728363 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7c668-246d-4c9f-91b3-e4ef61b935b7 service placement] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:27:22.738686 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7c668-246d-4c9f-91b3-e4ef61b935b7 service placement] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:27:22.742280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c5e48-e92f-4b77-ba30-15c74aadb853 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:27:22.742922 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c5e48-e92f-4b77-ba30-15c74aadb853 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:27:22.743200 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c5e48-e92f-4b77-ba30-15c74aadb853 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:27:22.743471 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c5e48-e92f-4b77-ba30-15c74aadb853 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:27:22.752740 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96c5e48-e92f-4b77-ba30-15c74aadb853 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:27:22.754171 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96c5e48-e92f-4b77-ba30-15c74aadb853 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:27:22.780864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1137/2254] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:27:22 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:27:32.778260 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=13 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:28:32.874787 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5983b80-c338-47a3-96d7-b92463563c4f service nova] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:28:32.876473 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a5983b80-c338-47a3-96d7-b92463563c4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:28:32.891530 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5983b80-c338-47a3-96d7-b92463563c4f service nova] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:28:32.897003 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c3b49-06fe-44dd-81ef-f99337cdad8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:28:32.897603 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c3b49-06fe-44dd-81ef-f99337cdad8c service nova] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:28:32.897876 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c3b49-06fe-44dd-81ef-f99337cdad8c service nova] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:28:32.898148 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c3b49-06fe-44dd-81ef-f99337cdad8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:28:32.900023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-f15c3b49-06fe-44dd-81ef-f99337cdad8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:28:32.975172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c3b49-06fe-44dd-81ef-f99337cdad8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:28:32.976784 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c3b49-06fe-44dd-81ef-f99337cdad8c service nova] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:28:32.986929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1118/2255] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:28:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:28:42.981720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=33 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:24.144673 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c5e48-e92f-4b77-ba30-15c74aadb853 service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:24.146505 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-b96c5e48-e92f-4b77-ba30-15c74aadb853 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:24.172295 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c5e48-e92f-4b77-ba30-15c74aadb853 service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:24.176039 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c9029-47b8-486d-a06b-7ca433ebfd98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:24.176656 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c9029-47b8-486d-a06b-7ca433ebfd98 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:24.176932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c9029-47b8-486d-a06b-7ca433ebfd98 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:24.177203 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c9029-47b8-486d-a06b-7ca433ebfd98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:24.187181 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c9029-47b8-486d-a06b-7ca433ebfd98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:24.188708 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c9029-47b8-486d-a06b-7ca433ebfd98 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:24.217921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1138/2256] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:29:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:29:26.529512 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c3b49-06fe-44dd-81ef-f99337cdad8c service nova] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:26.541853 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c3b49-06fe-44dd-81ef-f99337cdad8c service nova] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:26.548215 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29984c-ec4f-4149-857d-1e32baadcd2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:26.548916 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29984c-ec4f-4149-857d-1e32baadcd2c service nova] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:26.549253 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29984c-ec4f-4149-857d-1e32baadcd2c service nova] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:26.549525 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29984c-ec4f-4149-857d-1e32baadcd2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:26.559152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29984c-ec4f-4149-857d-1e32baadcd2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:26.561094 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29984c-ec4f-4149-857d-1e32baadcd2c service nova] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:26.572807 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1119/2257] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:29:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:29:32.251199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c9029-47b8-486d-a06b-7ca433ebfd98 service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:32.260467 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c9029-47b8-486d-a06b-7ca433ebfd98 service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:32.264667 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a973e64-730d-4079-a01c-785d1e3b4148 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:32.265259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a973e64-730d-4079-a01c-785d1e3b4148 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:32.265527 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a973e64-730d-4079-a01c-785d1e3b4148 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:32.265794 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a973e64-730d-4079-a01c-785d1e3b4148 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:32.267538 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-0a973e64-730d-4079-a01c-785d1e3b4148 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:32.351006 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a973e64-730d-4079-a01c-785d1e3b4148 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:32.352430 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a973e64-730d-4079-a01c-785d1e3b4148 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:32.371285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1139/2258] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:29:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:29:36.181774 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29984c-ec4f-4149-857d-1e32baadcd2c service nova] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:36.194579 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29984c-ec4f-4149-857d-1e32baadcd2c service nova] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:36.199880 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e07c1f-0d7d-4444-95e5-35f56c6ffda5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3db46a4b4e74cc0bb2a7bd6f0e99105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:36.201274 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e07c1f-0d7d-4444-95e5-35f56c6ffda5 service placement] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:36.201558 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e07c1f-0d7d-4444-95e5-35f56c6ffda5 service placement] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:36.201822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e07c1f-0d7d-4444-95e5-35f56c6ffda5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:36.203660 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-61e07c1f-0d7d-4444-95e5-35f56c6ffda5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:36.216351 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e07c1f-0d7d-4444-95e5-35f56c6ffda5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:36.217916 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e07c1f-0d7d-4444-95e5-35f56c6ffda5 service placement] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:36.227867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1120/2259] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:29:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:29:36.742603 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a973e64-730d-4079-a01c-785d1e3b4148 service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:36.744105 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-0a973e64-730d-4079-a01c-785d1e3b4148 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:36.794921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a973e64-730d-4079-a01c-785d1e3b4148 service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:36.794921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e656ca0-38a4-4275-b4d4-44f1b59f44f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:36.794921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e656ca0-38a4-4275-b4d4-44f1b59f44f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:36.794921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e656ca0-38a4-4275-b4d4-44f1b59f44f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:36.796402 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e656ca0-38a4-4275-b4d4-44f1b59f44f0 admin admin] PATH_INFO: `/v3/users/927bd33801e34715a78b58b268e28c4b` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:36.796402 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e656ca0-38a4-4275-b4d4-44f1b59f44f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=927bd33801e34715a78b58b268e28c4b)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:36.796402 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e656ca0-38a4-4275-b4d4-44f1b59f44f0 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:37.202424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-0e656ca0-38a4-4275-b4d4-44f1b59f44f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '927bd33801e34715a78b58b268e28c4b'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:29:37.214822 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-0e656ca0-38a4-4275-b4d4-44f1b59f44f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '927bd33801e34715a78b58b268e28c4b'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:29:37.222988 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-0e656ca0-38a4-4275-b4d4-44f1b59f44f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '927bd33801e34715a78b58b268e28c4b'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:29:37.312179 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-0e656ca0-38a4-4275-b4d4-44f1b59f44f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '927bd33801e34715a78b58b268e28c4b'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:29:37.312820 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-0e656ca0-38a4-4275-b4d4-44f1b59f44f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '927bd33801e34715a78b58b268e28c4b'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:29:37.314571 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:37.320502 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1140/2260] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:29:36 2022] DELETE /identity/v3/users/927bd33801e34715a78b58b268e28c4b => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:29:37.606187 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e07c1f-0d7d-4444-95e5-35f56c6ffda5 service placement] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:37.617274 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e07c1f-0d7d-4444-95e5-35f56c6ffda5 service placement] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:37.682428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc379917-9a58-4afb-a244-d37be450bbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:37.683085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc379917-9a58-4afb-a244-d37be450bbec admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:37.683364 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc379917-9a58-4afb-a244-d37be450bbec admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:37.683636 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc379917-9a58-4afb-a244-d37be450bbec admin admin] PATH_INFO: `/v3/projects/b4c01c0f214342aa88e4f71862604dd4` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:37.684793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc379917-9a58-4afb-a244-d37be450bbec admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4c01c0f214342aa88e4f71862604dd4)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:37.686764 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc379917-9a58-4afb-a244-d37be450bbec admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:37.798914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-fc379917-9a58-4afb-a244-d37be450bbec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4c01c0f214342aa88e4f71862604dd4', 'request_id': 'req-fc379917-9a58-4afb-a244-d37be450bbec'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:29:37.805263 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-fc379917-9a58-4afb-a244-d37be450bbec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4c01c0f214342aa88e4f71862604dd4', 'request_id': 'req-fc379917-9a58-4afb-a244-d37be450bbec'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:29:37.810838 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-fc379917-9a58-4afb-a244-d37be450bbec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4c01c0f214342aa88e4f71862604dd4', 'request_id': 'req-fc379917-9a58-4afb-a244-d37be450bbec'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:29:37.878071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-fc379917-9a58-4afb-a244-d37be450bbec admin admin] The token cache is being invalidate because project b4c01c0f214342aa88e4f71862604dd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:29:37.878292 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-fc379917-9a58-4afb-a244-d37be450bbec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:29:37.878915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-fc379917-9a58-4afb-a244-d37be450bbec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:29:37.880436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1121/2261] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:29:37 2022] DELETE /identity/v3/projects/b4c01c0f214342aa88e4f71862604dd4 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:29:37.886815 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65128d98-b296-4133-9df8-243b6c8c9a39 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:37.887286 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65128d98-b296-4133-9df8-243b6c8c9a39 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:37.887415 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65128d98-b296-4133-9df8-243b6c8c9a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:37.947252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-65128d98-b296-4133-9df8-243b6c8c9a39 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:37.948460 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-65128d98-b296-4133-9df8-243b6c8c9a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:38.009054 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1141/2262] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:29:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:29:38.017003 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e5f62-8849-49dd-bcdf-5d6bfb8abb80 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:38.017289 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e5f62-8849-49dd-bcdf-5d6bfb8abb80 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:38.017562 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e5f62-8849-49dd-bcdf-5d6bfb8abb80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:38.036936 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-447e5f62-8849-49dd-bcdf-5d6bfb8abb80 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:38.038149 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-447e5f62-8849-49dd-bcdf-5d6bfb8abb80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:38.046355 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1122/2263] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:29:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:29:38.051405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65128d98-b296-4133-9df8-243b6c8c9a39 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:38.060896 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65128d98-b296-4133-9df8-243b6c8c9a39 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:38.066164 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea153c1-5fb0-4e69-a18a-27ff088c5023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:38.066794 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea153c1-5fb0-4e69-a18a-27ff088c5023 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:38.067068 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea153c1-5fb0-4e69-a18a-27ff088c5023 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:38.067333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea153c1-5fb0-4e69-a18a-27ff088c5023 admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:38.067971 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea153c1-5fb0-4e69-a18a-27ff088c5023 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:29:38.068268 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea153c1-5fb0-4e69-a18a-27ff088c5023 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:38.069851 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea153c1-5fb0-4e69-a18a-27ff088c5023 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:38.081316 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1142/2264] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:29:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:29:38.085632 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e5f62-8849-49dd-bcdf-5d6bfb8abb80 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:38.096243 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e5f62-8849-49dd-bcdf-5d6bfb8abb80 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:38.101048 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56cead7-5e2b-4207-8555-4b172cb3092d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:38.101648 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56cead7-5e2b-4207-8555-4b172cb3092d admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:38.101921 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56cead7-5e2b-4207-8555-4b172cb3092d admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:38.102190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56cead7-5e2b-4207-8555-4b172cb3092d admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:38.102854 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56cead7-5e2b-4207-8555-4b172cb3092d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:38.104738 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56cead7-5e2b-4207-8555-4b172cb3092d admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:38.205304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1123/2265] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:29:38 2022] POST /identity/v3/projects => generated 371 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:29:38.209531 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea153c1-5fb0-4e69-a18a-27ff088c5023 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:38.220628 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea153c1-5fb0-4e69-a18a-27ff088c5023 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:38.275787 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b7d3e6-183d-4374-be97-7f1f8b6918b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:38.276373 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7d3e6-183d-4374-be97-7f1f8b6918b3 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:38.276648 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7d3e6-183d-4374-be97-7f1f8b6918b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:38.276912 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7d3e6-183d-4374-be97-7f1f8b6918b3 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:38.277540 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b7d3e6-183d-4374-be97-7f1f8b6918b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:38.279641 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b7d3e6-183d-4374-be97-7f1f8b6918b3 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:38.365204 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1143/2266] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:29:38 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:29:38.369241 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56cead7-5e2b-4207-8555-4b172cb3092d admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:38.380257 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56cead7-5e2b-4207-8555-4b172cb3092d admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:38.385085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d1f96-1fa7-42b2-aa7f-60208bab8cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:38.385762 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d1f96-1fa7-42b2-aa7f-60208bab8cd2 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:38.386104 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d1f96-1fa7-42b2-aa7f-60208bab8cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:38.386478 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d1f96-1fa7-42b2-aa7f-60208bab8cd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:38.387222 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079d1f96-1fa7-42b2-aa7f-60208bab8cd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:38.388983 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079d1f96-1fa7-42b2-aa7f-60208bab8cd2 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:38.399731 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1124/2267] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:29:38.404002 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b7d3e6-183d-4374-be97-7f1f8b6918b3 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:38.414492 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b7d3e6-183d-4374-be97-7f1f8b6918b3 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:38.419182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f3a37-57c7-49d3-af6b-57d7310007c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:38.419779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f3a37-57c7-49d3-af6b-57d7310007c2 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:38.420057 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f3a37-57c7-49d3-af6b-57d7310007c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:38.420328 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f3a37-57c7-49d3-af6b-57d7310007c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:38.420967 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f3a37-57c7-49d3-af6b-57d7310007c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:38.423526 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f3a37-57c7-49d3-af6b-57d7310007c2 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:38.434254 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1144/2268] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:29:38.438524 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d1f96-1fa7-42b2-aa7f-60208bab8cd2 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:38.449016 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d1f96-1fa7-42b2-aa7f-60208bab8cd2 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:38.454193 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326352cb-bc17-45b3-8e5d-24d41c2740f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:38.454874 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326352cb-bc17-45b3-8e5d-24d41c2740f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:38.455148 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326352cb-bc17-45b3-8e5d-24d41c2740f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:38.455477 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326352cb-bc17-45b3-8e5d-24d41c2740f9 admin admin] PATH_INFO: `/v3/projects/a368e821847243caa3e8c0cc34fc4769/users/24bbc03d3c80452ebe5cc63e1014b5ea/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:38.467916 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:38.471602 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326352cb-bc17-45b3-8e5d-24d41c2740f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a368e821847243caa3e8c0cc34fc4769, user_id=24bbc03d3c80452ebe5cc63e1014b5ea, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:38.474285 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326352cb-bc17-45b3-8e5d-24d41c2740f9 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:38.645053 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1125/2269] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:29:38 2022] PUT /identity/v3/projects/a368e821847243caa3e8c0cc34fc4769/users/24bbc03d3c80452ebe5cc63e1014b5ea/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:29:38.657469 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e18d9-0711-43e0-80d5-fa8d2a33974d None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:38.657751 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e18d9-0711-43e0-80d5-fa8d2a33974d None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:38.658054 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e18d9-0711-43e0-80d5-fa8d2a33974d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:38.685515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-ae7e18d9-0711-43e0-80d5-fa8d2a33974d None None] MFA Rules not processed for user `24bbc03d3c80452ebe5cc63e1014b5ea`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:38.686783 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-ae7e18d9-0711-43e0-80d5-fa8d2a33974d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:38.708208 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:38.752467 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1145/2270] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:29:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:29:38.756886 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326352cb-bc17-45b3-8e5d-24d41c2740f9 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:38.821001 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326352cb-bc17-45b3-8e5d-24d41c2740f9 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:38.825878 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab917aa-5de5-44d2-be11-aecdfe266502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:38.826513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab917aa-5de5-44d2-be11-aecdfe266502 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:38.826794 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab917aa-5de5-44d2-be11-aecdfe266502 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:38.827064 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab917aa-5de5-44d2-be11-aecdfe266502 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:38.827711 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab917aa-5de5-44d2-be11-aecdfe266502 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:38.830261 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab917aa-5de5-44d2-be11-aecdfe266502 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:39.000016 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1126/2271] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:29:38 2022] POST /identity/v3/projects => generated 371 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:29:39.004413 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e18d9-0711-43e0-80d5-fa8d2a33974d None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:39.015086 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e18d9-0711-43e0-80d5-fa8d2a33974d None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:39.071012 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c61dcb-25f3-4d69-b911-c00bde4bbee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:39.071613 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c61dcb-25f3-4d69-b911-c00bde4bbee5 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:39.071888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c61dcb-25f3-4d69-b911-c00bde4bbee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:39.072156 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c61dcb-25f3-4d69-b911-c00bde4bbee5 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:39.072775 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c61dcb-25f3-4d69-b911-c00bde4bbee5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:39.074707 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c61dcb-25f3-4d69-b911-c00bde4bbee5 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:39.159004 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1146/2272] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:29:39 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:29:39.162973 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab917aa-5de5-44d2-be11-aecdfe266502 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:39.173699 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab917aa-5de5-44d2-be11-aecdfe266502 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:39.178331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef2710-2a91-4b87-94aa-760773b625c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:39.179036 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef2710-2a91-4b87-94aa-760773b625c9 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:39.179314 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef2710-2a91-4b87-94aa-760773b625c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:39.179587 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef2710-2a91-4b87-94aa-760773b625c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:39.180228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ef2710-2a91-4b87-94aa-760773b625c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:39.181806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ef2710-2a91-4b87-94aa-760773b625c9 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:39.193319 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1127/2273] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:29:39.197461 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c61dcb-25f3-4d69-b911-c00bde4bbee5 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:39.209182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c61dcb-25f3-4d69-b911-c00bde4bbee5 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:39.214177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53611e3e-89f6-4e69-9eeb-a081dbf287aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:39.214856 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53611e3e-89f6-4e69-9eeb-a081dbf287aa admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:39.215132 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53611e3e-89f6-4e69-9eeb-a081dbf287aa admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:39.215400 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53611e3e-89f6-4e69-9eeb-a081dbf287aa admin admin] PATH_INFO: `/v3/projects/e829e37fbc0d4726bcf3030d2fe5cc22/users/d8f01b1961d24515915fa16f295e4899/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:39.230463 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53611e3e-89f6-4e69-9eeb-a081dbf287aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=e829e37fbc0d4726bcf3030d2fe5cc22, user_id=d8f01b1961d24515915fa16f295e4899, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:39.233199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53611e3e-89f6-4e69-9eeb-a081dbf287aa admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:39.330261 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1147/2274] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:29:39 2022] PUT /identity/v3/projects/e829e37fbc0d4726bcf3030d2fe5cc22/users/d8f01b1961d24515915fa16f295e4899/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:29:39.334194 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef2710-2a91-4b87-94aa-760773b625c9 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:39.345004 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef2710-2a91-4b87-94aa-760773b625c9 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:39.400858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8407da9-c304-4943-9c9d-4034c8dc12f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:39.401471 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8407da9-c304-4943-9c9d-4034c8dc12f6 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:39.401754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8407da9-c304-4943-9c9d-4034c8dc12f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:39.402043 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8407da9-c304-4943-9c9d-4034c8dc12f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:39.402837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8407da9-c304-4943-9c9d-4034c8dc12f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:39.404489 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8407da9-c304-4943-9c9d-4034c8dc12f6 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:39.415565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1128/2275] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:29:39.420160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53611e3e-89f6-4e69-9eeb-a081dbf287aa admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:39.431038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53611e3e-89f6-4e69-9eeb-a081dbf287aa admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:39.435694 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac8d19-fd87-4ada-a45a-a467bfaeab2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:39.436344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac8d19-fd87-4ada-a45a-a467bfaeab2c admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:39.436619 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac8d19-fd87-4ada-a45a-a467bfaeab2c admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:39.437574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac8d19-fd87-4ada-a45a-a467bfaeab2c admin admin] PATH_INFO: `/v3/projects/e829e37fbc0d4726bcf3030d2fe5cc22/users/d8f01b1961d24515915fa16f295e4899/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:39.439194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac8d19-fd87-4ada-a45a-a467bfaeab2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e829e37fbc0d4726bcf3030d2fe5cc22, user_id=d8f01b1961d24515915fa16f295e4899, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:39.441990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac8d19-fd87-4ada-a45a-a467bfaeab2c admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:39.458966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1148/2276] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:29:39 2022] PUT /identity/v3/projects/e829e37fbc0d4726bcf3030d2fe5cc22/users/d8f01b1961d24515915fa16f295e4899/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:29:39.470416 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a5884-9018-49e0-b689-73e1839b0825 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:39.470760 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a5884-9018-49e0-b689-73e1839b0825 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:39.470975 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a5884-9018-49e0-b689-73e1839b0825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:39.500913 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-950a5884-9018-49e0-b689-73e1839b0825 None None] MFA Rules not processed for user `d8f01b1961d24515915fa16f295e4899`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:39.536336 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-950a5884-9018-49e0-b689-73e1839b0825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:39.552383 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:39.576503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1129/2277] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:29:39 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:29:49.449269 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=61,keystone:INSERT=5 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:49.774600 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=6 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:51.375924 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac8d19-fd87-4ada-a45a-a467bfaeab2c admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:51.377763 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-69ac8d19-fd87-4ada-a45a-a467bfaeab2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:51.437749 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac8d19-fd87-4ada-a45a-a467bfaeab2c admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:51.441999 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26dc1f-0d1c-4c41-8891-6982e3f5b1e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:51.442632 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26dc1f-0d1c-4c41-8891-6982e3f5b1e0 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:51.442913 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26dc1f-0d1c-4c41-8891-6982e3f5b1e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:51.443185 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26dc1f-0d1c-4c41-8891-6982e3f5b1e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:51.444982 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-ca26dc1f-0d1c-4c41-8891-6982e3f5b1e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:51.522505 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26dc1f-0d1c-4c41-8891-6982e3f5b1e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:51.523977 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26dc1f-0d1c-4c41-8891-6982e3f5b1e0 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:51.548204 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1149/2278] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:29:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:29:51.590823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a5884-9018-49e0-b689-73e1839b0825 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:51.600470 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a5884-9018-49e0-b689-73e1839b0825 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:51.604531 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f356e-6d7c-4071-9d68-09777a8b77c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:51.605123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f356e-6d7c-4071-9d68-09777a8b77c9 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:51.605396 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f356e-6d7c-4071-9d68-09777a8b77c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:51.605664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f356e-6d7c-4071-9d68-09777a8b77c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:51.615399 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089f356e-6d7c-4071-9d68-09777a8b77c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:51.616809 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089f356e-6d7c-4071-9d68-09777a8b77c9 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:51.626416 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1130/2279] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:29:51 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:29:52.125936 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68b75d-4609-4879-a351-0764b06854e4 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:52.126220 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68b75d-4609-4879-a351-0764b06854e4 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:52.126533 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68b75d-4609-4879-a351-0764b06854e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:52.154713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-2b68b75d-4609-4879-a351-0764b06854e4 None None] MFA Rules not processed for user `d5d0010f584249b4a22fd31634c9fd8b`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:29:52.156001 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2b68b75d-4609-4879-a351-0764b06854e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:52.212191 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1150/2280] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:29:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:29:52.522820 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f356e-6d7c-4071-9d68-09777a8b77c9 service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:52.524255 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-089f356e-6d7c-4071-9d68-09777a8b77c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:52.594569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f356e-6d7c-4071-9d68-09777a8b77c9 service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:52.599162 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2e3a7-a0a0-4b7d-8e77-4e020598a736 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:52.599836 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2e3a7-a0a0-4b7d-8e77-4e020598a736 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:52.600114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2e3a7-a0a0-4b7d-8e77-4e020598a736 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:52.600383 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2e3a7-a0a0-4b7d-8e77-4e020598a736 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:52.602208 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-c4e2e3a7-a0a0-4b7d-8e77-4e020598a736 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:52.612041 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e2e3a7-a0a0-4b7d-8e77-4e020598a736 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:52.613507 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e2e3a7-a0a0-4b7d-8e77-4e020598a736 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:52.622028 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1131/2281] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:29:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:29:53.039578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68b75d-4609-4879-a351-0764b06854e4 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:53.041043 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2b68b75d-4609-4879-a351-0764b06854e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:53.087406 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:29:53.117710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68b75d-4609-4879-a351-0764b06854e4 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:53.123069 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439696b-aeb7-436f-85b7-81e49bc0bd6d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3db46a4b4e74cc0bb2a7bd6f0e99105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:53.123850 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439696b-aeb7-436f-85b7-81e49bc0bd6d service placement] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:53.124412 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439696b-aeb7-436f-85b7-81e49bc0bd6d service placement] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:53.124679 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439696b-aeb7-436f-85b7-81e49bc0bd6d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:53.126498 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-1439696b-aeb7-436f-85b7-81e49bc0bd6d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:53.138500 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1439696b-aeb7-436f-85b7-81e49bc0bd6d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:53.140087 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1439696b-aeb7-436f-85b7-81e49bc0bd6d service placement] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:53.149718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1151/2282] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:29:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:29:56.563003 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2e3a7-a0a0-4b7d-8e77-4e020598a736 service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:29:56.564444 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-c4e2e3a7-a0a0-4b7d-8e77-4e020598a736 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:56.575473 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2e3a7-a0a0-4b7d-8e77-4e020598a736 service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:29:56.579261 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042c76c-bbd3-4550-89b0-6bcfafa4ce81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:29:56.579892 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042c76c-bbd3-4550-89b0-6bcfafa4ce81 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:29:56.580222 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042c76c-bbd3-4550-89b0-6bcfafa4ce81 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:29:56.580536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042c76c-bbd3-4550-89b0-6bcfafa4ce81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:29:56.582275 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-5042c76c-bbd3-4550-89b0-6bcfafa4ce81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:29:56.593702 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042c76c-bbd3-4550-89b0-6bcfafa4ce81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:29:56.595217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042c76c-bbd3-4550-89b0-6bcfafa4ce81 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:29:56.603373 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1132/2283] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:29:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:30:03.145772 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=16 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:03.763719 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439696b-aeb7-436f-85b7-81e49bc0bd6d service placement] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:03.765314 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-1439696b-aeb7-436f-85b7-81e49bc0bd6d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:03.840153 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439696b-aeb7-436f-85b7-81e49bc0bd6d service placement] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:03.845973 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57883df0-4ff4-4684-a078-35276278e9dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:03.846746 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57883df0-4ff4-4684-a078-35276278e9dc service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:03.847204 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57883df0-4ff4-4684-a078-35276278e9dc service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:03.847535 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57883df0-4ff4-4684-a078-35276278e9dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:03.849318 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-57883df0-4ff4-4684-a078-35276278e9dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:03.862753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57883df0-4ff4-4684-a078-35276278e9dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:03.865045 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57883df0-4ff4-4684-a078-35276278e9dc service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:03.876186 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1152/2284] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:30:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:30:06.599818 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=43 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:13.871594 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=34 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:17.812289 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042c76c-bbd3-4550-89b0-6bcfafa4ce81 service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:17.821771 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042c76c-bbd3-4550-89b0-6bcfafa4ce81 service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:17.825591 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d5774-1e51-4267-9929-cd1edbeea92f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:17.826478 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d5774-1e51-4267-9929-cd1edbeea92f service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:17.826760 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d5774-1e51-4267-9929-cd1edbeea92f service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:17.827032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d5774-1e51-4267-9929-cd1edbeea92f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:17.828797 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-b88d5774-1e51-4267-9929-cd1edbeea92f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:17.858892 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d5774-1e51-4267-9929-cd1edbeea92f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:17.860311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d5774-1e51-4267-9929-cd1edbeea92f service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:17.872248 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1133/2285] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:30:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:30:19.005075 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57883df0-4ff4-4684-a078-35276278e9dc service nova] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:30:19.016216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57883df0-4ff4-4684-a078-35276278e9dc service nova] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:30:19.019907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00aedc-991c-4471-b829-07260c98a786 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:30:19.020503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00aedc-991c-4471-b829-07260c98a786 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:30:19.020780 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00aedc-991c-4471-b829-07260c98a786 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:30:19.021053 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00aedc-991c-4471-b829-07260c98a786 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:30:19.024196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2c00aedc-991c-4471-b829-07260c98a786 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:30:19.038372 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00aedc-991c-4471-b829-07260c98a786 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:30:19.039898 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00aedc-991c-4471-b829-07260c98a786 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:30:19.051015 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1153/2286] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:30:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:30:27.867018 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=11 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:30:29.046173 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=6 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:24.030945 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbcfe4-3e87-4bb8-afd4-d7aa7d1c6eab None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:24.034223 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbcfe4-3e87-4bb8-afd4-d7aa7d1c6eab None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:24.034223 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbcfe4-3e87-4bb8-afd4-d7aa7d1c6eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:24.091528 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-8ccbcfe4-3e87-4bb8-afd4-d7aa7d1c6eab None None] MFA Rules not processed for user `7796ebff70054de090e02c58bf7000f4`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:24.092774 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-8ccbcfe4-3e87-4bb8-afd4-d7aa7d1c6eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:24.158140 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1134/2287] 10.222.0.16 () {58 vars in 967 bytes} [Wed Aug 31 11:31:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:31:24.200784 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00aedc-991c-4471-b829-07260c98a786 service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:24.202227 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2c00aedc-991c-4471-b829-07260c98a786 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:24.291275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00aedc-991c-4471-b829-07260c98a786 service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:24.296405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab3fd3d-5b7d-4202-9191-eeb580c983ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8abc7c439a04622896b34ed0a7b8492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:24.297045 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3fd3d-5b7d-4202-9191-eeb580c983ed service swift] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:24.297324 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3fd3d-5b7d-4202-9191-eeb580c983ed service swift] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:24.297591 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3fd3d-5b7d-4202-9191-eeb580c983ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:24.306210 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab3fd3d-5b7d-4202-9191-eeb580c983ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:24.307810 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab3fd3d-5b7d-4202-9191-eeb580c983ed service swift] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:24.315708 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1154/2288] 10.222.0.16 () {60 vars in 1384 bytes} [Wed Aug 31 11:31:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:31:25.652140 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbcfe4-3e87-4bb8-afd4-d7aa7d1c6eab None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:25.662011 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbcfe4-3e87-4bb8-afd4-d7aa7d1c6eab None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:25.666700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16006b21-3c14-4fe2-af5e-1200c092bf5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:25.667434 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16006b21-3c14-4fe2-af5e-1200c092bf5e service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:25.667795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16006b21-3c14-4fe2-af5e-1200c092bf5e service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:25.668128 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16006b21-3c14-4fe2-af5e-1200c092bf5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:25.670199 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-16006b21-3c14-4fe2-af5e-1200c092bf5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:25.735963 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16006b21-3c14-4fe2-af5e-1200c092bf5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:25.737588 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16006b21-3c14-4fe2-af5e-1200c092bf5e service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:25.748594 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1135/2289] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:31:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:31:28.639806 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4c0c7-4f8d-41d5-8f8a-1feceff3c1c2 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:28.640294 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4c0c7-4f8d-41d5-8f8a-1feceff3c1c2 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:28.640525 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4c0c7-4f8d-41d5-8f8a-1feceff3c1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:28.666967 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-a5d4c0c7-4f8d-41d5-8f8a-1feceff3c1c2 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:28.668992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a5d4c0c7-4f8d-41d5-8f8a-1feceff3c1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:28.682568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1155/2290] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:31:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:31:28.695408 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131346e-b66b-411b-8eb5-0318011d881a None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:28.695892 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131346e-b66b-411b-8eb5-0318011d881a None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:28.696003 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131346e-b66b-411b-8eb5-0318011d881a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:28.717683 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-5131346e-b66b-411b-8eb5-0318011d881a None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:28.719289 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-5131346e-b66b-411b-8eb5-0318011d881a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:28.729229 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1136/2291] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:31:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:31:28.733331 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4c0c7-4f8d-41d5-8f8a-1feceff3c1c2 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:28.745607 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4c0c7-4f8d-41d5-8f8a-1feceff3c1c2 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:28.750461 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca52f85-030e-4060-9d54-eeea5470d80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:28.751189 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca52f85-030e-4060-9d54-eeea5470d80b admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:28.751471 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca52f85-030e-4060-9d54-eeea5470d80b admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:28.751740 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca52f85-030e-4060-9d54-eeea5470d80b admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:28.752418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca52f85-030e-4060-9d54-eeea5470d80b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:31:28.752716 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca52f85-030e-4060-9d54-eeea5470d80b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:28.754330 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca52f85-030e-4060-9d54-eeea5470d80b admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:28.769523 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1156/2292] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:31:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:31:28.778124 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131346e-b66b-411b-8eb5-0318011d881a None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:28.788882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131346e-b66b-411b-8eb5-0318011d881a None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:28.794697 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b5689-f619-4e6e-a47f-2c3dab9065cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:28.797061 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b5689-f619-4e6e-a47f-2c3dab9065cb admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:28.797061 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b5689-f619-4e6e-a47f-2c3dab9065cb admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:28.797061 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b5689-f619-4e6e-a47f-2c3dab9065cb admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:28.797061 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3b5689-f619-4e6e-a47f-2c3dab9065cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:28.799906 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3b5689-f619-4e6e-a47f-2c3dab9065cb admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:28.924888 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1137/2293] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:31:28 2022] POST /identity/v3/projects => generated 373 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:31:28.929816 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca52f85-030e-4060-9d54-eeea5470d80b admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:28.941055 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca52f85-030e-4060-9d54-eeea5470d80b admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:29.013089 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf250df-c682-40bc-9f3a-5c028caaf3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:29.013650 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf250df-c682-40bc-9f3a-5c028caaf3f7 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:29.013927 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf250df-c682-40bc-9f3a-5c028caaf3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:29.014196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf250df-c682-40bc-9f3a-5c028caaf3f7 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:29.014896 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf250df-c682-40bc-9f3a-5c028caaf3f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:29.016814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf250df-c682-40bc-9f3a-5c028caaf3f7 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:29.153533 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1157/2294] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:31:28 2022] POST /identity/v3/users => generated 287 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:31:29.157783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b5689-f619-4e6e-a47f-2c3dab9065cb admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:29.169324 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b5689-f619-4e6e-a47f-2c3dab9065cb admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:29.174030 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d8045-8e8c-4222-9210-1a7b0cca3f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:29.174656 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d8045-8e8c-4222-9210-1a7b0cca3f38 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:29.174980 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d8045-8e8c-4222-9210-1a7b0cca3f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:29.175193 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d8045-8e8c-4222-9210-1a7b0cca3f38 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:29.175858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d8045-8e8c-4222-9210-1a7b0cca3f38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:29.177440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d8045-8e8c-4222-9210-1a7b0cca3f38 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:29.187733 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1138/2295] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:31:29.191802 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf250df-c682-40bc-9f3a-5c028caaf3f7 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:29.202005 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf250df-c682-40bc-9f3a-5c028caaf3f7 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:29.206631 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e3e1c-a308-47f8-863e-ad11a6cfd92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:29.207220 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e3e1c-a308-47f8-863e-ad11a6cfd92a admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:29.207492 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e3e1c-a308-47f8-863e-ad11a6cfd92a admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:29.207758 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e3e1c-a308-47f8-863e-ad11a6cfd92a admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:29.208390 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e3e1c-a308-47f8-863e-ad11a6cfd92a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:29.210642 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e3e1c-a308-47f8-863e-ad11a6cfd92a admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:29.221359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1158/2296] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:31:29.227097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d8045-8e8c-4222-9210-1a7b0cca3f38 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:29.237796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d8045-8e8c-4222-9210-1a7b0cca3f38 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:29.254912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f7ee1-a9ab-4c28-b1c3-7d0de1a0902a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:29.255576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f7ee1-a9ab-4c28-b1c3-7d0de1a0902a admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:29.255853 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f7ee1-a9ab-4c28-b1c3-7d0de1a0902a admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:29.256125 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f7ee1-a9ab-4c28-b1c3-7d0de1a0902a admin admin] PATH_INFO: `/v3/projects/39682f18e68946efb4a82c883a01dd5f/users/095a406c2a9e40e2a57b5eb298640276/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:29.271700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f7ee1-a9ab-4c28-b1c3-7d0de1a0902a admin admin] RBAC: Authorizing `identity:create_grant(project_id=39682f18e68946efb4a82c883a01dd5f, user_id=095a406c2a9e40e2a57b5eb298640276, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:29.274911 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f7ee1-a9ab-4c28-b1c3-7d0de1a0902a admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:29.277117 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:29.386682 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1139/2297] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:31:29 2022] PUT /identity/v3/projects/39682f18e68946efb4a82c883a01dd5f/users/095a406c2a9e40e2a57b5eb298640276/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:31:29.396342 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789b9e8-b29f-4cd1-93d0-157c52bf0250 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:29.396717 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789b9e8-b29f-4cd1-93d0-157c52bf0250 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:29.396978 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789b9e8-b29f-4cd1-93d0-157c52bf0250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:29.417137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-b789b9e8-b29f-4cd1-93d0-157c52bf0250 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:29.418435 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-b789b9e8-b29f-4cd1-93d0-157c52bf0250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:29.446058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:29.482062 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1159/2298] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:31:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:31:29.492209 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f7ee1-a9ab-4c28-b1c3-7d0de1a0902a admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:29.547676 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f7ee1-a9ab-4c28-b1c3-7d0de1a0902a admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:29.551327 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c036be9-1491-40c1-9dea-5d2204aa1a17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:29.551916 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c036be9-1491-40c1-9dea-5d2204aa1a17 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:29.552188 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c036be9-1491-40c1-9dea-5d2204aa1a17 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:29.552456 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c036be9-1491-40c1-9dea-5d2204aa1a17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:29.561490 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c036be9-1491-40c1-9dea-5d2204aa1a17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:29.562955 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c036be9-1491-40c1-9dea-5d2204aa1a17 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:29.573249 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1140/2299] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:31:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:31:35.311513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431022f-d755-4955-9f58-5906ff94187b None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:35.312128 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431022f-d755-4955-9f58-5906ff94187b None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:35.312128 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431022f-d755-4955-9f58-5906ff94187b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:35.344337 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-5431022f-d755-4955-9f58-5906ff94187b None None] MFA Rules not processed for user `095a406c2a9e40e2a57b5eb298640276`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:35.345707 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-5431022f-d755-4955-9f58-5906ff94187b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:35.420554 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1160/2300] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:31:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:31:35.425097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c036be9-1491-40c1-9dea-5d2204aa1a17 service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:35.442560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c036be9-1491-40c1-9dea-5d2204aa1a17 service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:35.443325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e299d-66d7-405e-9a31-8ca79371de22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:35.443458 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e299d-66d7-405e-9a31-8ca79371de22 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:35.443655 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e299d-66d7-405e-9a31-8ca79371de22 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:35.443922 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e299d-66d7-405e-9a31-8ca79371de22 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:35.444615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e299d-66d7-405e-9a31-8ca79371de22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:35.446561 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e299d-66d7-405e-9a31-8ca79371de22 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:35.535321 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1141/2301] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:31:35 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:31:35.535937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431022f-d755-4955-9f58-5906ff94187b None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:35.549022 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431022f-d755-4955-9f58-5906ff94187b None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:35.641936 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5215f-3f05-4d66-a2a6-9f007ab28fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:35.642498 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5215f-3f05-4d66-a2a6-9f007ab28fa7 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:35.642816 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5215f-3f05-4d66-a2a6-9f007ab28fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:35.643087 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5215f-3f05-4d66-a2a6-9f007ab28fa7 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:35.643745 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff5215f-3f05-4d66-a2a6-9f007ab28fa7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:35.645707 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff5215f-3f05-4d66-a2a6-9f007ab28fa7 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:35.756083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1161/2302] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:31:35 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:31:35.760872 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e299d-66d7-405e-9a31-8ca79371de22 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:35.771820 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e299d-66d7-405e-9a31-8ca79371de22 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:35.777910 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6318b-c027-454f-99b6-e79396c045e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:35.778581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6318b-c027-454f-99b6-e79396c045e5 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:35.778864 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6318b-c027-454f-99b6-e79396c045e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:35.779137 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6318b-c027-454f-99b6-e79396c045e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:35.779801 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e6318b-c027-454f-99b6-e79396c045e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:35.781471 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e6318b-c027-454f-99b6-e79396c045e5 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:35.795666 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1142/2303] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:31:35.800202 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5215f-3f05-4d66-a2a6-9f007ab28fa7 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:35.811861 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5215f-3f05-4d66-a2a6-9f007ab28fa7 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:35.816863 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37224ccd-a9ab-4c7d-b5a5-7aa31982a17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:35.817473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37224ccd-a9ab-4c7d-b5a5-7aa31982a17c admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:35.817752 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37224ccd-a9ab-4c7d-b5a5-7aa31982a17c admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:35.818026 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37224ccd-a9ab-4c7d-b5a5-7aa31982a17c admin admin] PATH_INFO: `/v3/projects/2d77dc6509724be7ae2c30224a34622b/users/e5746817cf1c46ea80ca1c6157b034c8/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:35.837921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37224ccd-a9ab-4c7d-b5a5-7aa31982a17c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d77dc6509724be7ae2c30224a34622b, user_id=e5746817cf1c46ea80ca1c6157b034c8, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:35.841515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37224ccd-a9ab-4c7d-b5a5-7aa31982a17c admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:35.922991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1162/2304] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:31:35 2022] PUT /identity/v3/projects/2d77dc6509724be7ae2c30224a34622b/users/e5746817cf1c46ea80ca1c6157b034c8/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:31:35.926854 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6318b-c027-454f-99b6-e79396c045e5 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:35.937657 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6318b-c027-454f-99b6-e79396c045e5 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:36.001525 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73e976-099b-488e-ad9d-91c9b4562f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:36.002280 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73e976-099b-488e-ad9d-91c9b4562f52 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:36.002590 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73e976-099b-488e-ad9d-91c9b4562f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:36.002860 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73e976-099b-488e-ad9d-91c9b4562f52 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:36.003532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d73e976-099b-488e-ad9d-91c9b4562f52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:36.005115 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d73e976-099b-488e-ad9d-91c9b4562f52 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:36.016347 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1143/2305] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:31:36.020616 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37224ccd-a9ab-4c7d-b5a5-7aa31982a17c admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:36.032534 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37224ccd-a9ab-4c7d-b5a5-7aa31982a17c admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:36.037355 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec25255-dee7-4655-9289-16b3c5910243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:36.038251 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec25255-dee7-4655-9289-16b3c5910243 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:36.038629 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec25255-dee7-4655-9289-16b3c5910243 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:36.038902 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec25255-dee7-4655-9289-16b3c5910243 admin admin] PATH_INFO: `/v3/projects/2d77dc6509724be7ae2c30224a34622b/users/e5746817cf1c46ea80ca1c6157b034c8/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:36.040841 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec25255-dee7-4655-9289-16b3c5910243 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d77dc6509724be7ae2c30224a34622b, user_id=e5746817cf1c46ea80ca1c6157b034c8, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:36.043686 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec25255-dee7-4655-9289-16b3c5910243 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:36.057166 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1163/2306] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:31:36 2022] PUT /identity/v3/projects/2d77dc6509724be7ae2c30224a34622b/users/e5746817cf1c46ea80ca1c6157b034c8/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:31:42.580625 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a215422-0819-4867-a414-6b8d8efebf99 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:42.581569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a215422-0819-4867-a414-6b8d8efebf99 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:42.581966 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a215422-0819-4867-a414-6b8d8efebf99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:42.627299 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-5a215422-0819-4867-a414-6b8d8efebf99 None None] MFA Rules not processed for user `e5746817cf1c46ea80ca1c6157b034c8`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:42.628632 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-5a215422-0819-4867-a414-6b8d8efebf99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:42.674044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:42.713410 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1144/2307] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:31:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:31:42.729199 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec25255-dee7-4655-9289-16b3c5910243 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:42.730975 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-5ec25255-dee7-4655-9289-16b3c5910243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:42.747576 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:42.828626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec25255-dee7-4655-9289-16b3c5910243 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:42.834178 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3cd35b-f497-4a15-a9eb-46cabd81d6e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:42.835379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3cd35b-f497-4a15-a9eb-46cabd81d6e8 service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:42.835683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3cd35b-f497-4a15-a9eb-46cabd81d6e8 service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:42.836064 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3cd35b-f497-4a15-a9eb-46cabd81d6e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:42.896029 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3cd35b-f497-4a15-a9eb-46cabd81d6e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:42.898128 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3cd35b-f497-4a15-a9eb-46cabd81d6e8 service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:42.907662 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1164/2308] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:31:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:31:45.931471 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28a7f3-87a2-46ad-8463-280930ee2e1b None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:45.936912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28a7f3-87a2-46ad-8463-280930ee2e1b None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:45.936912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28a7f3-87a2-46ad-8463-280930ee2e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:45.951844 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-3a28a7f3-87a2-46ad-8463-280930ee2e1b None None] MFA Rules not processed for user `095a406c2a9e40e2a57b5eb298640276`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:31:45.953065 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-3a28a7f3-87a2-46ad-8463-280930ee2e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:45.960585 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1145/2309] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:31:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:31:48.794782 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3cd35b-f497-4a15-a9eb-46cabd81d6e8 service nova] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:31:48.796281 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-ab3cd35b-f497-4a15-a9eb-46cabd81d6e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:48.886540 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3cd35b-f497-4a15-a9eb-46cabd81d6e8 service nova] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:31:48.891536 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce0519-e4a9-433c-9218-878193825919 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3db46a4b4e74cc0bb2a7bd6f0e99105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:31:48.892159 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce0519-e4a9-433c-9218-878193825919 service placement] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:31:48.892432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce0519-e4a9-433c-9218-878193825919 service placement] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:31:48.892698 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce0519-e4a9-433c-9218-878193825919 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:31:48.894454 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-34ce0519-e4a9-433c-9218-878193825919 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:31:48.906508 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce0519-e4a9-433c-9218-878193825919 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:31:48.908079 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce0519-e4a9-433c-9218-878193825919 service placement] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:31:48.917777 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1165/2310] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:31:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:31:55.946678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=17 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:31:58.913688 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:24.944311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a28a7f3-87a2-46ad-8463-280930ee2e1b None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:24.945827 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-3a28a7f3-87a2-46ad-8463-280930ee2e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:33:25.016220 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a28a7f3-87a2-46ad-8463-280930ee2e1b None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:25.020000 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cee81e-ffa6-4fe8-a7ba-eb0632350487 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:25.020620 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cee81e-ffa6-4fe8-a7ba-eb0632350487 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:25.020899 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cee81e-ffa6-4fe8-a7ba-eb0632350487 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:25.021174 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cee81e-ffa6-4fe8-a7ba-eb0632350487 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:25.097111 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cee81e-ffa6-4fe8-a7ba-eb0632350487 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:25.098617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cee81e-ffa6-4fe8-a7ba-eb0632350487 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:25.110808 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1146/2311] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:33:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:33:25.554873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce0519-e4a9-433c-9218-878193825919 service placement] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:25.574769 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce0519-e4a9-433c-9218-878193825919 service placement] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:25.578619 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6d1ac-22cf-408e-852f-41df13781332 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:25.579220 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6d1ac-22cf-408e-852f-41df13781332 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:25.579490 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6d1ac-22cf-408e-852f-41df13781332 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:25.579759 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6d1ac-22cf-408e-852f-41df13781332 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:25.581528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-fbb6d1ac-22cf-408e-852f-41df13781332 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:33:25.658426 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb6d1ac-22cf-408e-852f-41df13781332 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:25.660293 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb6d1ac-22cf-408e-852f-41df13781332 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:25.669522 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1166/2312] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:33:25 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:33:26.605198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cee81e-ffa6-4fe8-a7ba-eb0632350487 service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:26.667213 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cee81e-ffa6-4fe8-a7ba-eb0632350487 service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:26.671563 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622a9c0-ba48-49fa-81cb-c0df2910d6f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:26.672178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622a9c0-ba48-49fa-81cb-c0df2910d6f2 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:26.672458 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622a9c0-ba48-49fa-81cb-c0df2910d6f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:26.672728 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622a9c0-ba48-49fa-81cb-c0df2910d6f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:26.674535 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-8622a9c0-ba48-49fa-81cb-c0df2910d6f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:33:26.723117 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:26.738860 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8622a9c0-ba48-49fa-81cb-c0df2910d6f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:26.740325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8622a9c0-ba48-49fa-81cb-c0df2910d6f2 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:26.750242 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1147/2313] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:33:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:33:35.665391 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=38 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:36.756560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=7 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:33:53.623241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6d1ac-22cf-408e-852f-41df13781332 service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:33:53.643550 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6d1ac-22cf-408e-852f-41df13781332 service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:33:53.647365 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3affb2a1-ac60-4765-a29b-73585bd4a906 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:33:53.648004 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affb2a1-ac60-4765-a29b-73585bd4a906 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:33:53.648284 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affb2a1-ac60-4765-a29b-73585bd4a906 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:33:53.648555 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affb2a1-ac60-4765-a29b-73585bd4a906 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:33:53.650470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-3affb2a1-ac60-4765-a29b-73585bd4a906 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:33:53.727681 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3affb2a1-ac60-4765-a29b-73585bd4a906 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:33:53.729555 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3affb2a1-ac60-4765-a29b-73585bd4a906 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:33:53.749298 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1167/2314] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:33:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:34:03.747152 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=40 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:26.529276 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622a9c0-ba48-49fa-81cb-c0df2910d6f2 service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:35:26.544237 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622a9c0-ba48-49fa-81cb-c0df2910d6f2 service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:35:26.549030 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7854dec8-fbc7-4df5-b9b0-784a10f0d4f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:35:26.549660 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7854dec8-fbc7-4df5-b9b0-784a10f0d4f7 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:26.549862 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7854dec8-fbc7-4df5-b9b0-784a10f0d4f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:26.550138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7854dec8-fbc7-4df5-b9b0-784a10f0d4f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:26.643869 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7854dec8-fbc7-4df5-b9b0-784a10f0d4f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:35:26.645255 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7854dec8-fbc7-4df5-b9b0-784a10f0d4f7 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:35:26.656341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1148/2315] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:35:26 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:35:26.807093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3affb2a1-ac60-4765-a29b-73585bd4a906 service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:35:26.824325 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3affb2a1-ac60-4765-a29b-73585bd4a906 service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:35:26.828712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7513c8-0000-42a7-9b8d-504b7cfda51f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:35:26.829316 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7513c8-0000-42a7-9b8d-504b7cfda51f service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:26.829589 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7513c8-0000-42a7-9b8d-504b7cfda51f service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:26.829854 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7513c8-0000-42a7-9b8d-504b7cfda51f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:26.831946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-fc7513c8-0000-42a7-9b8d-504b7cfda51f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:26.907745 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7513c8-0000-42a7-9b8d-504b7cfda51f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:35:26.909138 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7513c8-0000-42a7-9b8d-504b7cfda51f service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:35:26.919442 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1168/2316] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:35:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:35:27.017044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7854dec8-fbc7-4df5-b9b0-784a10f0d4f7 service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:35:27.025833 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7854dec8-fbc7-4df5-b9b0-784a10f0d4f7 service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:35:27.030468 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798dc22f-50a4-4824-af04-0338c8fcca3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:35:27.031078 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798dc22f-50a4-4824-af04-0338c8fcca3a service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:27.031359 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798dc22f-50a4-4824-af04-0338c8fcca3a service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:27.031634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798dc22f-50a4-4824-af04-0338c8fcca3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:27.109956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798dc22f-50a4-4824-af04-0338c8fcca3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:35:27.110690 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798dc22f-50a4-4824-af04-0338c8fcca3a service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:35:27.121394 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1149/2317] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:35:27 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:35:36.914415 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=33 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:37.116521 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=58 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:35:54.292339 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7513c8-0000-42a7-9b8d-504b7cfda51f service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:35:54.313115 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7513c8-0000-42a7-9b8d-504b7cfda51f service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:35:54.317085 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db4a31-6b10-4af4-9108-c9e7ed7cb72a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:35:54.317701 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db4a31-6b10-4af4-9108-c9e7ed7cb72a service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:54.317983 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db4a31-6b10-4af4-9108-c9e7ed7cb72a service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:54.318253 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db4a31-6b10-4af4-9108-c9e7ed7cb72a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:54.320078 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-88db4a31-6b10-4af4-9108-c9e7ed7cb72a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:54.345535 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798dc22f-50a4-4824-af04-0338c8fcca3a service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:35:54.354998 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798dc22f-50a4-4824-af04-0338c8fcca3a service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:35:54.359516 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c392e75-5895-44ae-8f87-77359869b99c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:35:54.360125 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c392e75-5895-44ae-8f87-77359869b99c service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:54.360405 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c392e75-5895-44ae-8f87-77359869b99c service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:54.360679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c392e75-5895-44ae-8f87-77359869b99c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:54.368081 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c392e75-5895-44ae-8f87-77359869b99c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:35:54.369572 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c392e75-5895-44ae-8f87-77359869b99c service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:35:54.378618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1150/2318] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:35:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:35:54.379421 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c392e75-5895-44ae-8f87-77359869b99c service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:35:54.388153 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c392e75-5895-44ae-8f87-77359869b99c service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:35:54.392654 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db4a31-6b10-4af4-9108-c9e7ed7cb72a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:35:54.394018 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db4a31-6b10-4af4-9108-c9e7ed7cb72a service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:35:54.394244 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fed712-5d3d-4244-9060-ff2de667171e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:35:54.394895 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fed712-5d3d-4244-9060-ff2de667171e service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:54.395226 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fed712-5d3d-4244-9060-ff2de667171e service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:54.395445 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fed712-5d3d-4244-9060-ff2de667171e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:54.398176 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-d0fed712-5d3d-4244-9060-ff2de667171e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:54.405007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1169/2319] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:35:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:35:54.411251 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fed712-5d3d-4244-9060-ff2de667171e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:35:54.412694 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fed712-5d3d-4244-9060-ff2de667171e service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:35:54.425673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1151/2320] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:35:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:35:59.557654 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95d0da-8d6a-4f88-891d-eb456218ba14 None None] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:59.558625 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95d0da-8d6a-4f88-891d-eb456218ba14 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:59.559222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95d0da-8d6a-4f88-891d-eb456218ba14 None None] PATH_INFO: `/` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:59.561475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1170/2321] 10.222.0.16 () {54 vars in 921 bytes} [Wed Aug 31 11:35:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:35:59.575376 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9ed32-e518-48a4-8969-6c6fa3508be9 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:59.575939 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9ed32-e518-48a4-8969-6c6fa3508be9 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:59.576204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9ed32-e518-48a4-8969-6c6fa3508be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:59.642165 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95d0da-8d6a-4f88-891d-eb456218ba14 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:35:59.646359 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-b6e9ed32-e518-48a4-8969-6c6fa3508be9 None None] MFA Rules not processed for user `f3db46a4b4e74cc0bb2a7bd6f0e99105`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:35:59.647646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-b6e9ed32-e518-48a4-8969-6c6fa3508be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:59.657583 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95d0da-8d6a-4f88-891d-eb456218ba14 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:35:59.658827 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1152/2322] 10.222.0.16 () {58 vars in 1018 bytes} [Wed Aug 31 11:35:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:35:59.662962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c5af1-8e69-4098-b3f8-f1e8d28ccfd2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3db46a4b4e74cc0bb2a7bd6f0e99105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:35:59.663806 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c5af1-8e69-4098-b3f8-f1e8d28ccfd2 service placement] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:59.664296 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c5af1-8e69-4098-b3f8-f1e8d28ccfd2 service placement] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:59.665214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c5af1-8e69-4098-b3f8-f1e8d28ccfd2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:59.667540 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-503c5af1-8e69-4098-b3f8-f1e8d28ccfd2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:59.670853 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e9ed32-e518-48a4-8969-6c6fa3508be9 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:35:59.682735 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c5af1-8e69-4098-b3f8-f1e8d28ccfd2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:35:59.684409 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e9ed32-e518-48a4-8969-6c6fa3508be9 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:35:59.685841 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c5af1-8e69-4098-b3f8-f1e8d28ccfd2 service placement] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:35:59.691791 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3619cc3-e640-4b97-9f12-f48bdfbe2178 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3db46a4b4e74cc0bb2a7bd6f0e99105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:35:59.692691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3619cc3-e640-4b97-9f12-f48bdfbe2178 service placement] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:59.692975 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3619cc3-e640-4b97-9f12-f48bdfbe2178 service placement] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:59.693244 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3619cc3-e640-4b97-9f12-f48bdfbe2178 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:59.700868 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1171/2323] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:35:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:35:59.704866 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3619cc3-e640-4b97-9f12-f48bdfbe2178 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:35:59.706498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3619cc3-e640-4b97-9f12-f48bdfbe2178 service placement] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:35:59.718953 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1153/2324] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:35:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:35:59.787754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c5af1-8e69-4098-b3f8-f1e8d28ccfd2 service placement] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:35:59.789255 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-503c5af1-8e69-4098-b3f8-f1e8d28ccfd2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:59.809578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c5af1-8e69-4098-b3f8-f1e8d28ccfd2 service placement] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:35:59.814835 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412cf7b-1327-4faf-9d36-339cc5e4d3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:35:59.815329 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412cf7b-1327-4faf-9d36-339cc5e4d3a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:59.816380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412cf7b-1327-4faf-9d36-339cc5e4d3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:59.816847 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412cf7b-1327-4faf-9d36-339cc5e4d3a9 admin admin] PATH_INFO: `/v3/users/8ab477a1ab0d4e61b9649b62ee5295a0` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:59.820214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412cf7b-1327-4faf-9d36-339cc5e4d3a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ab477a1ab0d4e61b9649b62ee5295a0)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:35:59.822601 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3619cc3-e640-4b97-9f12-f48bdfbe2178 service placement] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:35:59.824351 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412cf7b-1327-4faf-9d36-339cc5e4d3a9 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:35:59.824969 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-e3619cc3-e640-4b97-9f12-f48bdfbe2178 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:35:59.847556 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3619cc3-e640-4b97-9f12-f48bdfbe2178 service placement] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:35:59.852777 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fad683-1908-468b-b509-7fa41c081a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:35:59.853447 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fad683-1908-468b-b509-7fa41c081a93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:35:59.853726 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fad683-1908-468b-b509-7fa41c081a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:35:59.853997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fad683-1908-468b-b509-7fa41c081a93 admin admin] PATH_INFO: `/v3/users/5d1ce9150ba543b1b06f8edb5a545af0` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:35:59.855190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fad683-1908-468b-b509-7fa41c081a93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d1ce9150ba543b1b06f8edb5a545af0)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:35:59.858072 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fad683-1908-468b-b509-7fa41c081a93 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:00.022863 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a412cf7b-1327-4faf-9d36-339cc5e4d3a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ab477a1ab0d4e61b9649b62ee5295a0'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:36:00.035790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a412cf7b-1327-4faf-9d36-339cc5e4d3a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ab477a1ab0d4e61b9649b62ee5295a0'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:36:00.042964 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a412cf7b-1327-4faf-9d36-339cc5e4d3a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ab477a1ab0d4e61b9649b62ee5295a0'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:36:00.057273 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-20fad683-1908-468b-b509-7fa41c081a93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d1ce9150ba543b1b06f8edb5a545af0'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:36:00.150354 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a412cf7b-1327-4faf-9d36-339cc5e4d3a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ab477a1ab0d4e61b9649b62ee5295a0'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:36:00.151034 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a412cf7b-1327-4faf-9d36-339cc5e4d3a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ab477a1ab0d4e61b9649b62ee5295a0'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:36:00.158761 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1172/2325] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:35:59 2022] DELETE /identity/v3/users/8ab477a1ab0d4e61b9649b62ee5295a0 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:36:00.216900 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-20fad683-1908-468b-b509-7fa41c081a93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d1ce9150ba543b1b06f8edb5a545af0'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:36:00.217555 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-20fad683-1908-468b-b509-7fa41c081a93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d1ce9150ba543b1b06f8edb5a545af0'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:36:00.218271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-20fad683-1908-468b-b509-7fa41c081a93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d1ce9150ba543b1b06f8edb5a545af0'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:36:00.234413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-20fad683-1908-468b-b509-7fa41c081a93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d1ce9150ba543b1b06f8edb5a545af0'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:36:00.249753 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1154/2326] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:35:59 2022] DELETE /identity/v3/users/5d1ce9150ba543b1b06f8edb5a545af0 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:36:00.522040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412cf7b-1327-4faf-9d36-339cc5e4d3a9 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:00.527096 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a412cf7b-1327-4faf-9d36-339cc5e4d3a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:00.554802 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fad683-1908-468b-b509-7fa41c081a93 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:00.556377 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-20fad683-1908-468b-b509-7fa41c081a93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:00.590885 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:00.610252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412cf7b-1327-4faf-9d36-339cc5e4d3a9 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:00.635159 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fad683-1908-468b-b509-7fa41c081a93 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:00.640679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca31dc5-d5c2-4620-bb6e-c0d9e0eee832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:00.641353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca31dc5-d5c2-4620-bb6e-c0d9e0eee832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:00.641628 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca31dc5-d5c2-4620-bb6e-c0d9e0eee832 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:00.641901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca31dc5-d5c2-4620-bb6e-c0d9e0eee832 admin admin] PATH_INFO: `/v3/projects/58687b56ab864698b5c107cbea52a8dc` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:00.643238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca31dc5-d5c2-4620-bb6e-c0d9e0eee832 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58687b56ab864698b5c107cbea52a8dc)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:00.645165 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca31dc5-d5c2-4620-bb6e-c0d9e0eee832 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:00.663195 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:00.677152 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497a5db-fb88-4bbb-9718-f8fbada76b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:00.677812 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497a5db-fb88-4bbb-9718-f8fbada76b1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:00.678087 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497a5db-fb88-4bbb-9718-f8fbada76b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:00.678355 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497a5db-fb88-4bbb-9718-f8fbada76b1a admin admin] PATH_INFO: `/v3/projects/644e7abc63304b8d950b786654696f21` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:00.679522 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e497a5db-fb88-4bbb-9718-f8fbada76b1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=644e7abc63304b8d950b786654696f21)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:00.681412 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e497a5db-fb88-4bbb-9718-f8fbada76b1a admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:00.759584 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-8ca31dc5-d5c2-4620-bb6e-c0d9e0eee832 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58687b56ab864698b5c107cbea52a8dc', 'request_id': 'req-8ca31dc5-d5c2-4620-bb6e-c0d9e0eee832'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:36:00.766156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-8ca31dc5-d5c2-4620-bb6e-c0d9e0eee832 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58687b56ab864698b5c107cbea52a8dc', 'request_id': 'req-8ca31dc5-d5c2-4620-bb6e-c0d9e0eee832'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:36:00.771724 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-8ca31dc5-d5c2-4620-bb6e-c0d9e0eee832 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58687b56ab864698b5c107cbea52a8dc', 'request_id': 'req-8ca31dc5-d5c2-4620-bb6e-c0d9e0eee832'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:36:00.827971 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e497a5db-fb88-4bbb-9718-f8fbada76b1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '644e7abc63304b8d950b786654696f21', 'request_id': 'req-e497a5db-fb88-4bbb-9718-f8fbada76b1a'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:36:00.912139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-8ca31dc5-d5c2-4620-bb6e-c0d9e0eee832 admin admin] The token cache is being invalidate because project 58687b56ab864698b5c107cbea52a8dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:36:00.912424 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-8ca31dc5-d5c2-4620-bb6e-c0d9e0eee832 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:36:00.913070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-8ca31dc5-d5c2-4620-bb6e-c0d9e0eee832 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:36:00.914610 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1155/2327] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:36:00 2022] DELETE /identity/v3/projects/58687b56ab864698b5c107cbea52a8dc => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:36:00.920799 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266337ac-47b9-4a20-abaa-b830eb95d3b4 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:00.924031 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266337ac-47b9-4a20-abaa-b830eb95d3b4 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:00.924031 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266337ac-47b9-4a20-abaa-b830eb95d3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:00.942249 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-266337ac-47b9-4a20-abaa-b830eb95d3b4 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:00.943503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-266337ac-47b9-4a20-abaa-b830eb95d3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:00.978923 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e497a5db-fb88-4bbb-9718-f8fbada76b1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '644e7abc63304b8d950b786654696f21', 'request_id': 'req-e497a5db-fb88-4bbb-9718-f8fbada76b1a'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:36:00.985231 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e497a5db-fb88-4bbb-9718-f8fbada76b1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '644e7abc63304b8d950b786654696f21', 'request_id': 'req-e497a5db-fb88-4bbb-9718-f8fbada76b1a'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:36:00.996788 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e497a5db-fb88-4bbb-9718-f8fbada76b1a admin admin] The token cache is being invalidate because project 644e7abc63304b8d950b786654696f21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:36:00.997159 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e497a5db-fb88-4bbb-9718-f8fbada76b1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:36:00.997806 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e497a5db-fb88-4bbb-9718-f8fbada76b1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:36:00.999340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1173/2328] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:36:00 2022] DELETE /identity/v3/projects/644e7abc63304b8d950b786654696f21 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:36:01.004986 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1156/2329] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:36:01.005860 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe02756-7cb5-4d83-9709-ec0fe5f8ed4f None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:01.006155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe02756-7cb5-4d83-9709-ec0fe5f8ed4f None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:01.006462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe02756-7cb5-4d83-9709-ec0fe5f8ed4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:01.013441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658772b-7517-470e-b1f6-88ba3bcf256d None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:01.013722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658772b-7517-470e-b1f6-88ba3bcf256d None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:01.013986 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658772b-7517-470e-b1f6-88ba3bcf256d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:01.029094 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-2fe02756-7cb5-4d83-9709-ec0fe5f8ed4f None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:01.030376 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2fe02756-7cb5-4d83-9709-ec0fe5f8ed4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:01.034661 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-0658772b-7517-470e-b1f6-88ba3bcf256d None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:01.036006 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-0658772b-7517-470e-b1f6-88ba3bcf256d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:01.038728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1174/2330] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:36:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:36:01.044924 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1157/2331] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:36:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:36:01.048933 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dd2bd-682b-4f65-a94e-54659d6df021 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:01.049138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0658772b-7517-470e-b1f6-88ba3bcf256d None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:01.049261 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dd2bd-682b-4f65-a94e-54659d6df021 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:01.049479 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dd2bd-682b-4f65-a94e-54659d6df021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:01.059577 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0658772b-7517-470e-b1f6-88ba3bcf256d None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:01.064934 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e337b-9a64-462c-8aa2-c74dc7071021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:01.065653 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e337b-9a64-462c-8aa2-c74dc7071021 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:01.065814 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e337b-9a64-462c-8aa2-c74dc7071021 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:01.066178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e337b-9a64-462c-8aa2-c74dc7071021 admin admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:01.066756 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e337b-9a64-462c-8aa2-c74dc7071021 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:36:01.067052 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e337b-9a64-462c-8aa2-c74dc7071021 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:01.068777 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e337b-9a64-462c-8aa2-c74dc7071021 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:01.071437 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-121dd2bd-682b-4f65-a94e-54659d6df021 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:01.072829 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-121dd2bd-682b-4f65-a94e-54659d6df021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:01.081365 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1175/2332] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:36:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:36:01.081933 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1158/2333] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:36:01.086374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121dd2bd-682b-4f65-a94e-54659d6df021 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:01.086596 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e337b-9a64-462c-8aa2-c74dc7071021 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:01.097313 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e337b-9a64-462c-8aa2-c74dc7071021 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:01.099963 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121dd2bd-682b-4f65-a94e-54659d6df021 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:01.102167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f53d2-9160-4d6d-bf1b-256b31c0d74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:01.102838 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f53d2-9160-4d6d-bf1b-256b31c0d74a admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:01.103105 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f53d2-9160-4d6d-bf1b-256b31c0d74a admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:01.103377 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f53d2-9160-4d6d-bf1b-256b31c0d74a admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:01.105985 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617f53d2-9160-4d6d-bf1b-256b31c0d74a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:01.106187 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2d1d1-ff07-4cd6-b74f-b4d393c0d237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:01.106187 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2d1d1-ff07-4cd6-b74f-b4d393c0d237 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:01.106645 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2d1d1-ff07-4cd6-b74f-b4d393c0d237 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:01.106804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617f53d2-9160-4d6d-bf1b-256b31c0d74a admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:01.106965 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2d1d1-ff07-4cd6-b74f-b4d393c0d237 admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:01.107743 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2d1d1-ff07-4cd6-b74f-b4d393c0d237 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:36:01.107992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2d1d1-ff07-4cd6-b74f-b4d393c0d237 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:01.109637 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2d1d1-ff07-4cd6-b74f-b4d393c0d237 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:01.123462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1176/2334] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:36:01.128073 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2d1d1-ff07-4cd6-b74f-b4d393c0d237 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:01.138957 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2d1d1-ff07-4cd6-b74f-b4d393c0d237 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:01.143814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80a3ee-af50-4069-a5d9-5a08c52d18ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:01.144416 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80a3ee-af50-4069-a5d9-5a08c52d18ba admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:01.144696 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80a3ee-af50-4069-a5d9-5a08c52d18ba admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:01.144970 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80a3ee-af50-4069-a5d9-5a08c52d18ba admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:01.145605 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80a3ee-af50-4069-a5d9-5a08c52d18ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:01.147574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80a3ee-af50-4069-a5d9-5a08c52d18ba admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:01.194045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1159/2335] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:36:01 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:36:01.198125 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f53d2-9160-4d6d-bf1b-256b31c0d74a admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:01.249963 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1177/2336] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:36:01 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:36:01.251079 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80a3ee-af50-4069-a5d9-5a08c52d18ba admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:01.267664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f53d2-9160-4d6d-bf1b-256b31c0d74a admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:01.273927 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:01.287000 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:01.315697 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80a3ee-af50-4069-a5d9-5a08c52d18ba admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:01.320642 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41298dc-6c59-459f-937b-e2dfc1bacdbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:01.321252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41298dc-6c59-459f-937b-e2dfc1bacdbc admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:01.321534 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41298dc-6c59-459f-937b-e2dfc1bacdbc admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:01.321816 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41298dc-6c59-459f-937b-e2dfc1bacdbc admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:01.322504 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41298dc-6c59-459f-937b-e2dfc1bacdbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:01.324597 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41298dc-6c59-459f-937b-e2dfc1bacdbc admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:01.331153 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c65ab-4ee6-4514-af0b-db3b1941d087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:01.333597 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c65ab-4ee6-4514-af0b-db3b1941d087 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:01.333597 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c65ab-4ee6-4514-af0b-db3b1941d087 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:01.333597 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c65ab-4ee6-4514-af0b-db3b1941d087 admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:01.333597 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c65ab-4ee6-4514-af0b-db3b1941d087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:01.334908 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c65ab-4ee6-4514-af0b-db3b1941d087 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:01.441841 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1178/2337] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:36:01 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:36:01.445956 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41298dc-6c59-459f-937b-e2dfc1bacdbc admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:01.456265 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41298dc-6c59-459f-937b-e2dfc1bacdbc admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:01.462717 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1160/2338] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:36:01 2022] POST /identity/v3/users => generated 283 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:36:01.462987 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ce2d1-df82-49f2-9f40-843b2c451595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:01.463817 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ce2d1-df82-49f2-9f40-843b2c451595 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:01.464167 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ce2d1-df82-49f2-9f40-843b2c451595 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:01.464479 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ce2d1-df82-49f2-9f40-843b2c451595 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:01.466703 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ce2d1-df82-49f2-9f40-843b2c451595 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:01.467313 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c65ab-4ee6-4514-af0b-db3b1941d087 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:01.468412 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ce2d1-df82-49f2-9f40-843b2c451595 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:01.477612 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c65ab-4ee6-4514-af0b-db3b1941d087 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:01.479375 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1179/2339] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:36:01.483736 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f439b0d-c3f0-4de9-b5bd-be0ed0b70a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:01.484358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f439b0d-c3f0-4de9-b5bd-be0ed0b70a3a admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:01.484632 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f439b0d-c3f0-4de9-b5bd-be0ed0b70a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:01.485636 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f439b0d-c3f0-4de9-b5bd-be0ed0b70a3a admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:01.485636 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f439b0d-c3f0-4de9-b5bd-be0ed0b70a3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:01.487787 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f439b0d-c3f0-4de9-b5bd-be0ed0b70a3a admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:01.488810 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ce2d1-df82-49f2-9f40-843b2c451595 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:01.498964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1161/2340] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:36:01.499748 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ce2d1-df82-49f2-9f40-843b2c451595 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:01.504044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f439b0d-c3f0-4de9-b5bd-be0ed0b70a3a admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:01.504907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93317535-4adf-49ca-bde7-a34d100ee31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:01.505553 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93317535-4adf-49ca-bde7-a34d100ee31c admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:01.505831 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93317535-4adf-49ca-bde7-a34d100ee31c admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:01.506152 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93317535-4adf-49ca-bde7-a34d100ee31c admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:01.506828 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93317535-4adf-49ca-bde7-a34d100ee31c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:01.508488 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93317535-4adf-49ca-bde7-a34d100ee31c admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:01.514818 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f439b0d-c3f0-4de9-b5bd-be0ed0b70a3a admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:01.519881 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773b135-9ea7-4020-bb5f-588885e0f104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:01.520527 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773b135-9ea7-4020-bb5f-588885e0f104 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:01.520675 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1180/2341] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:36:01.520803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773b135-9ea7-4020-bb5f-588885e0f104 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:01.521070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773b135-9ea7-4020-bb5f-588885e0f104 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:01.521720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773b135-9ea7-4020-bb5f-588885e0f104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:01.523364 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773b135-9ea7-4020-bb5f-588885e0f104 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:01.524790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93317535-4adf-49ca-bde7-a34d100ee31c admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:01.535397 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93317535-4adf-49ca-bde7-a34d100ee31c admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:01.536896 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1162/2342] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:36:01.540325 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88dc08f-dbf9-4d0c-b493-9b661c0cbc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:01.540991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88dc08f-dbf9-4d0c-b493-9b661c0cbc04 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:01.541183 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773b135-9ea7-4020-bb5f-588885e0f104 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:01.541304 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88dc08f-dbf9-4d0c-b493-9b661c0cbc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:01.541518 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88dc08f-dbf9-4d0c-b493-9b661c0cbc04 admin admin] PATH_INFO: `/v3/projects/2092c2eeb351400588f4098c0f733e21/users/deff6bcb6f9748c086170d40a1f8c30a/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:01.553046 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773b135-9ea7-4020-bb5f-588885e0f104 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:01.557801 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb42ca5-15d8-4dde-95c8-19075a78a1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:01.558494 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb42ca5-15d8-4dde-95c8-19075a78a1f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:01.558779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb42ca5-15d8-4dde-95c8-19075a78a1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:01.559031 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb42ca5-15d8-4dde-95c8-19075a78a1f6 admin admin] PATH_INFO: `/v3/projects/94607d6a3c184e5980dfd87f0d5462cf/users/e38560c102ef446c95892d546a2450af/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:01.559909 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88dc08f-dbf9-4d0c-b493-9b661c0cbc04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2092c2eeb351400588f4098c0f733e21, user_id=deff6bcb6f9748c086170d40a1f8c30a, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:01.562743 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88dc08f-dbf9-4d0c-b493-9b661c0cbc04 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:01.575668 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb42ca5-15d8-4dde-95c8-19075a78a1f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94607d6a3c184e5980dfd87f0d5462cf, user_id=e38560c102ef446c95892d546a2450af, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:01.580337 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb42ca5-15d8-4dde-95c8-19075a78a1f6 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:01.651141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1181/2343] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:36:01 2022] PUT /identity/v3/projects/2092c2eeb351400588f4098c0f733e21/users/deff6bcb6f9748c086170d40a1f8c30a/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:36:01.670252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78d05c-31bb-4570-b2ba-d500cafc6670 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:01.670634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78d05c-31bb-4570-b2ba-d500cafc6670 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:01.670908 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78d05c-31bb-4570-b2ba-d500cafc6670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:01.685623 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1163/2344] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:36:01 2022] PUT /identity/v3/projects/94607d6a3c184e5980dfd87f0d5462cf/users/e38560c102ef446c95892d546a2450af/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:36:01.704806 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-cc78d05c-31bb-4570-b2ba-d500cafc6670 None None] MFA Rules not processed for user `deff6bcb6f9748c086170d40a1f8c30a`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:01.705194 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714b164-22d7-4b27-9d1e-4887535335d8 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:01.705624 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714b164-22d7-4b27-9d1e-4887535335d8 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:01.705976 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714b164-22d7-4b27-9d1e-4887535335d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:01.706101 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-cc78d05c-31bb-4570-b2ba-d500cafc6670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:01.736419 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-0714b164-22d7-4b27-9d1e-4887535335d8 None None] MFA Rules not processed for user `e38560c102ef446c95892d546a2450af`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:01.737694 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-0714b164-22d7-4b27-9d1e-4887535335d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:01.776810 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1182/2345] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:36:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:36:01.790516 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78d05c-31bb-4570-b2ba-d500cafc6670 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:01.792042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-cc78d05c-31bb-4570-b2ba-d500cafc6670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:01.807018 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1164/2346] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:36:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:36:01.811430 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714b164-22d7-4b27-9d1e-4887535335d8 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:01.859808 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:01.861634 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:01.877798 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714b164-22d7-4b27-9d1e-4887535335d8 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:01.882642 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe04b1-5ed9-4048-9885-44d196b073bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:01.883259 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe04b1-5ed9-4048-9885-44d196b073bb admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:01.883540 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe04b1-5ed9-4048-9885-44d196b073bb admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:01.883835 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe04b1-5ed9-4048-9885-44d196b073bb admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:01.884482 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe04b1-5ed9-4048-9885-44d196b073bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:01.886462 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe04b1-5ed9-4048-9885-44d196b073bb admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:01.889960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78d05c-31bb-4570-b2ba-d500cafc6670 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:01.895776 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd808b0f-1786-4b5f-9dc3-a26dcf2caf91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:01.899555 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd808b0f-1786-4b5f-9dc3-a26dcf2caf91 service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:01.899555 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd808b0f-1786-4b5f-9dc3-a26dcf2caf91 service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:01.899555 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd808b0f-1786-4b5f-9dc3-a26dcf2caf91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:01.906620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd808b0f-1786-4b5f-9dc3-a26dcf2caf91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:01.908147 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd808b0f-1786-4b5f-9dc3-a26dcf2caf91 service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:01.917307 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1183/2347] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:36:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:36:02.000872 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1165/2348] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:36:01 2022] POST /identity/v3/projects => generated 365 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:36:02.005001 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd808b0f-1786-4b5f-9dc3-a26dcf2caf91 service nova] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:02.016703 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd808b0f-1786-4b5f-9dc3-a26dcf2caf91 service nova] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:02.077200 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553066c9-19cb-48e8-a823-5717a5508222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:02.077811 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553066c9-19cb-48e8-a823-5717a5508222 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:02.078092 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553066c9-19cb-48e8-a823-5717a5508222 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:02.078367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553066c9-19cb-48e8-a823-5717a5508222 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:02.079041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553066c9-19cb-48e8-a823-5717a5508222 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:02.081054 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553066c9-19cb-48e8-a823-5717a5508222 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:02.206163 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1184/2349] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:36:02 2022] POST /identity/v3/users => generated 289 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:36:02.210443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe04b1-5ed9-4048-9885-44d196b073bb admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:02.220804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe04b1-5ed9-4048-9885-44d196b073bb admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:02.225318 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077c7c3-6724-405f-8f34-3d2c1909c43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:02.225972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077c7c3-6724-405f-8f34-3d2c1909c43b admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:02.226246 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077c7c3-6724-405f-8f34-3d2c1909c43b admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:02.226554 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077c7c3-6724-405f-8f34-3d2c1909c43b admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:02.227203 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077c7c3-6724-405f-8f34-3d2c1909c43b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:02.228777 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077c7c3-6724-405f-8f34-3d2c1909c43b admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:02.238986 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1166/2350] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:36:02.243327 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553066c9-19cb-48e8-a823-5717a5508222 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:02.254292 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553066c9-19cb-48e8-a823-5717a5508222 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:02.258930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab176d-663a-4745-9ca1-411d9cc15e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:02.259573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab176d-663a-4745-9ca1-411d9cc15e46 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:02.259849 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab176d-663a-4745-9ca1-411d9cc15e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:02.260123 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab176d-663a-4745-9ca1-411d9cc15e46 admin admin] PATH_INFO: `/v3/projects/d86be09318244848a615ad51cf55ca76/users/7221971389bb45b7815f0a867dcc7ef4/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:02.275779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab176d-663a-4745-9ca1-411d9cc15e46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d86be09318244848a615ad51cf55ca76, user_id=7221971389bb45b7815f0a867dcc7ef4, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:02.279074 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab176d-663a-4745-9ca1-411d9cc15e46 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:02.347299 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1185/2351] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:36:02 2022] PUT /identity/v3/projects/d86be09318244848a615ad51cf55ca76/users/7221971389bb45b7815f0a867dcc7ef4/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:36:02.351528 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077c7c3-6724-405f-8f34-3d2c1909c43b admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:02.361432 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077c7c3-6724-405f-8f34-3d2c1909c43b admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:02.419442 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557708ad-0f35-46c9-838b-059aabda199d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:02.419963 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557708ad-0f35-46c9-838b-059aabda199d admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:02.420238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557708ad-0f35-46c9-838b-059aabda199d admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:02.420513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557708ad-0f35-46c9-838b-059aabda199d admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:02.421143 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557708ad-0f35-46c9-838b-059aabda199d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:02.422851 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557708ad-0f35-46c9-838b-059aabda199d admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:02.433247 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1167/2352] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:36:02.439168 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab176d-663a-4745-9ca1-411d9cc15e46 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:02.449949 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab176d-663a-4745-9ca1-411d9cc15e46 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:02.455188 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc1665-6a00-4273-a207-d0dd7efc8a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:02.455834 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc1665-6a00-4273-a207-d0dd7efc8a2d admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:02.456108 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc1665-6a00-4273-a207-d0dd7efc8a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:02.456374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc1665-6a00-4273-a207-d0dd7efc8a2d admin admin] PATH_INFO: `/v3/projects/d86be09318244848a615ad51cf55ca76/users/7221971389bb45b7815f0a867dcc7ef4/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:02.458298 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc1665-6a00-4273-a207-d0dd7efc8a2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d86be09318244848a615ad51cf55ca76, user_id=7221971389bb45b7815f0a867dcc7ef4, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:02.461217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc1665-6a00-4273-a207-d0dd7efc8a2d admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:02.472155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1186/2353] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:36:02 2022] PUT /identity/v3/projects/d86be09318244848a615ad51cf55ca76/users/7221971389bb45b7815f0a867dcc7ef4/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:36:02.484305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35fe38-edda-4819-b236-3962ba5e9d73 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:02.484581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35fe38-edda-4819-b236-3962ba5e9d73 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:02.484854 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35fe38-edda-4819-b236-3962ba5e9d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:02.513274 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-9b35fe38-edda-4819-b236-3962ba5e9d73 None None] MFA Rules not processed for user `7221971389bb45b7815f0a867dcc7ef4`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:36:02.514517 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-9b35fe38-edda-4819-b236-3962ba5e9d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:02.584475 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1168/2354] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:36:02 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:36:12.468300 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=55,keystone:INSERT=5 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:12.581206 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=82,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:36:55.534006 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc1665-6a00-4273-a207-d0dd7efc8a2d admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:36:55.535766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-bafc1665-6a00-4273-a207-d0dd7efc8a2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:55.607878 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc1665-6a00-4273-a207-d0dd7efc8a2d admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:36:55.612962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29a766-43e1-4451-a248-2a6f72cd796e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3db46a4b4e74cc0bb2a7bd6f0e99105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:36:55.614218 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29a766-43e1-4451-a248-2a6f72cd796e service placement] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:36:55.614529 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29a766-43e1-4451-a248-2a6f72cd796e service placement] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:36:55.614802 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29a766-43e1-4451-a248-2a6f72cd796e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:36:55.616521 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-ee29a766-43e1-4451-a248-2a6f72cd796e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:36:55.628982 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee29a766-43e1-4451-a248-2a6f72cd796e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:36:55.630612 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee29a766-43e1-4451-a248-2a6f72cd796e service placement] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:36:55.641616 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1187/2355] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:36:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:37:01.853867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35fe38-edda-4819-b236-3962ba5e9d73 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:01.916662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35fe38-edda-4819-b236-3962ba5e9d73 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:01.921336 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3a085-86db-480d-92b4-87e26c680f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:01.921967 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3a085-86db-480d-92b4-87e26c680f6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:01.922244 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3a085-86db-480d-92b4-87e26c680f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:01.922555 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3a085-86db-480d-92b4-87e26c680f6d admin admin] PATH_INFO: `/v3/users/deff6bcb6f9748c086170d40a1f8c30a` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:01.923682 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3a085-86db-480d-92b4-87e26c680f6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=deff6bcb6f9748c086170d40a1f8c30a)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:01.925599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3a085-86db-480d-92b4-87e26c680f6d admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:02.103784 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-bbf3a085-86db-480d-92b4-87e26c680f6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deff6bcb6f9748c086170d40a1f8c30a'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:37:02.174940 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-bbf3a085-86db-480d-92b4-87e26c680f6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deff6bcb6f9748c086170d40a1f8c30a'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:37:02.175600 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-bbf3a085-86db-480d-92b4-87e26c680f6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deff6bcb6f9748c086170d40a1f8c30a'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:37:02.176208 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-bbf3a085-86db-480d-92b4-87e26c680f6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deff6bcb6f9748c086170d40a1f8c30a'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:37:02.190198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-bbf3a085-86db-480d-92b4-87e26c680f6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deff6bcb6f9748c086170d40a1f8c30a'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:37:02.203252 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1169/2356] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:37:01 2022] DELETE /identity/v3/users/deff6bcb6f9748c086170d40a1f8c30a => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:37:02.208170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464caa1-b9d7-4762-9b18-6be3c3715df6 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:02.208697 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464caa1-b9d7-4762-9b18-6be3c3715df6 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:02.208697 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464caa1-b9d7-4762-9b18-6be3c3715df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:02.228925 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-e464caa1-b9d7-4762-9b18-6be3c3715df6 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:37:02.230262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-e464caa1-b9d7-4762-9b18-6be3c3715df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:02.290202 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1188/2357] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:37:02.300221 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3a085-86db-480d-92b4-87e26c680f6d admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:02.301685 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-bbf3a085-86db-480d-92b4-87e26c680f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:02.362038 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3a085-86db-480d-92b4-87e26c680f6d admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:02.365565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd52171b-c33b-4409-af54-1c19ce682a86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:02.366167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52171b-c33b-4409-af54-1c19ce682a86 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:02.366468 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52171b-c33b-4409-af54-1c19ce682a86 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:02.366740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52171b-c33b-4409-af54-1c19ce682a86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:02.376360 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd52171b-c33b-4409-af54-1c19ce682a86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:02.377787 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd52171b-c33b-4409-af54-1c19ce682a86 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:02.387878 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1170/2358] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:37:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:37:02.811926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e464caa1-b9d7-4762-9b18-6be3c3715df6 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:02.813512 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-e464caa1-b9d7-4762-9b18-6be3c3715df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:02.826256 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e464caa1-b9d7-4762-9b18-6be3c3715df6 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:02.831221 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11570a34-d67b-4499-8817-02f37d27966b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:02.832180 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11570a34-d67b-4499-8817-02f37d27966b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:02.832291 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11570a34-d67b-4499-8817-02f37d27966b admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:02.832565 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11570a34-d67b-4499-8817-02f37d27966b admin admin] PATH_INFO: `/v3/projects/2092c2eeb351400588f4098c0f733e21` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:02.833765 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11570a34-d67b-4499-8817-02f37d27966b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2092c2eeb351400588f4098c0f733e21)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:02.836378 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11570a34-d67b-4499-8817-02f37d27966b admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:02.936185 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-11570a34-d67b-4499-8817-02f37d27966b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2092c2eeb351400588f4098c0f733e21', 'request_id': 'req-11570a34-d67b-4499-8817-02f37d27966b'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:37:03.007315 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-11570a34-d67b-4499-8817-02f37d27966b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2092c2eeb351400588f4098c0f733e21', 'request_id': 'req-11570a34-d67b-4499-8817-02f37d27966b'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:37:03.015094 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-11570a34-d67b-4499-8817-02f37d27966b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2092c2eeb351400588f4098c0f733e21', 'request_id': 'req-11570a34-d67b-4499-8817-02f37d27966b'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:37:03.026301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-11570a34-d67b-4499-8817-02f37d27966b admin admin] The token cache is being invalidate because project 2092c2eeb351400588f4098c0f733e21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:37:03.026632 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-11570a34-d67b-4499-8817-02f37d27966b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:37:03.027275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-11570a34-d67b-4499-8817-02f37d27966b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:37:03.029105 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1189/2359] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:37:02 2022] DELETE /identity/v3/projects/2092c2eeb351400588f4098c0f733e21 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:37:03.033941 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bdd79-8774-4194-904e-ff936b2e4477 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:03.034447 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bdd79-8774-4194-904e-ff936b2e4477 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:03.034573 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bdd79-8774-4194-904e-ff936b2e4477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:03.054183 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-1c4bdd79-8774-4194-904e-ff936b2e4477 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:37:03.055544 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-1c4bdd79-8774-4194-904e-ff936b2e4477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:03.073088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:03.117241 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1171/2360] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:37:03.126050 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af870a02-3d5c-489d-af07-70ed00ae1421 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:03.126332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af870a02-3d5c-489d-af07-70ed00ae1421 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:03.126684 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af870a02-3d5c-489d-af07-70ed00ae1421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:03.147866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-af870a02-3d5c-489d-af07-70ed00ae1421 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:37:03.149094 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-af870a02-3d5c-489d-af07-70ed00ae1421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:03.157418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1190/2361] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:37:03.161898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4bdd79-8774-4194-904e-ff936b2e4477 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:03.172500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4bdd79-8774-4194-904e-ff936b2e4477 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:03.178090 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8031c73-974f-430c-b668-aa6c5bc18327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:03.178734 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8031c73-974f-430c-b668-aa6c5bc18327 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:03.179014 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8031c73-974f-430c-b668-aa6c5bc18327 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:03.179380 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8031c73-974f-430c-b668-aa6c5bc18327 admin admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:03.184681 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8031c73-974f-430c-b668-aa6c5bc18327 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:37:03.184681 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8031c73-974f-430c-b668-aa6c5bc18327 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:03.184681 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8031c73-974f-430c-b668-aa6c5bc18327 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:03.195658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1172/2362] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:37:03.200620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af870a02-3d5c-489d-af07-70ed00ae1421 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:03.210503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af870a02-3d5c-489d-af07-70ed00ae1421 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:03.215647 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cc6d3-f2a4-449a-9e82-fea97cfe487a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:03.216263 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cc6d3-f2a4-449a-9e82-fea97cfe487a admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:03.216538 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cc6d3-f2a4-449a-9e82-fea97cfe487a admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:03.216808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cc6d3-f2a4-449a-9e82-fea97cfe487a admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:03.217447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0cc6d3-f2a4-449a-9e82-fea97cfe487a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:03.219389 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0cc6d3-f2a4-449a-9e82-fea97cfe487a admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:03.224292 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:03.331674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1191/2363] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:37:03 2022] POST /identity/v3/projects => generated 381 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:37:03.336439 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8031c73-974f-430c-b668-aa6c5bc18327 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:03.347962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8031c73-974f-430c-b668-aa6c5bc18327 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:03.417019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef927ad-d8f4-4ba8-adee-7a7226a825a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:03.417636 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef927ad-d8f4-4ba8-adee-7a7226a825a7 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:03.417913 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef927ad-d8f4-4ba8-adee-7a7226a825a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:03.418182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef927ad-d8f4-4ba8-adee-7a7226a825a7 admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:03.419475 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef927ad-d8f4-4ba8-adee-7a7226a825a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:03.423724 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef927ad-d8f4-4ba8-adee-7a7226a825a7 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:03.522159 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1173/2364] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:37:03 2022] POST /identity/v3/users => generated 291 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:37:03.526245 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cc6d3-f2a4-449a-9e82-fea97cfe487a admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:03.538707 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cc6d3-f2a4-449a-9e82-fea97cfe487a admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:03.544125 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2498f4c-aa83-436a-abd9-b2f0436ae37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:03.545517 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2498f4c-aa83-436a-abd9-b2f0436ae37d admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:03.545801 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2498f4c-aa83-436a-abd9-b2f0436ae37d admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:03.546100 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2498f4c-aa83-436a-abd9-b2f0436ae37d admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:03.546837 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2498f4c-aa83-436a-abd9-b2f0436ae37d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:03.552932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2498f4c-aa83-436a-abd9-b2f0436ae37d admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:03.560012 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1192/2365] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:37:03.565949 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef927ad-d8f4-4ba8-adee-7a7226a825a7 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:03.578463 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef927ad-d8f4-4ba8-adee-7a7226a825a7 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:03.583547 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70091cc4-b408-4326-bb3c-34fbea29bc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:03.584282 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70091cc4-b408-4326-bb3c-34fbea29bc76 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:03.584566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70091cc4-b408-4326-bb3c-34fbea29bc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:03.585116 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70091cc4-b408-4326-bb3c-34fbea29bc76 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:03.585771 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70091cc4-b408-4326-bb3c-34fbea29bc76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:03.587471 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70091cc4-b408-4326-bb3c-34fbea29bc76 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:03.601985 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1174/2366] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:37:03.607257 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2498f4c-aa83-436a-abd9-b2f0436ae37d admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:03.617944 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2498f4c-aa83-436a-abd9-b2f0436ae37d admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:03.623160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff263b-3e66-4d22-94a0-53f7471db14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:03.628226 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff263b-3e66-4d22-94a0-53f7471db14b admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:03.628226 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff263b-3e66-4d22-94a0-53f7471db14b admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:03.628226 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff263b-3e66-4d22-94a0-53f7471db14b admin admin] PATH_INFO: `/v3/projects/c5578bf9341646b9b4665cc0c1b13250/users/b3d6118f7e3a4158bb456a07368ec961/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:03.640640 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aff263b-3e66-4d22-94a0-53f7471db14b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5578bf9341646b9b4665cc0c1b13250, user_id=b3d6118f7e3a4158bb456a07368ec961, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:03.643542 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aff263b-3e66-4d22-94a0-53f7471db14b admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:03.736579 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1193/2367] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:37:03 2022] PUT /identity/v3/projects/c5578bf9341646b9b4665cc0c1b13250/users/b3d6118f7e3a4158bb456a07368ec961/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:37:03.747706 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f337f-7b85-4f7e-b84a-870d369f2b93 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:03.748062 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f337f-7b85-4f7e-b84a-870d369f2b93 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:03.748371 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f337f-7b85-4f7e-b84a-870d369f2b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:03.780844 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-ae0f337f-7b85-4f7e-b84a-870d369f2b93 None None] MFA Rules not processed for user `b3d6118f7e3a4158bb456a07368ec961`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:37:03.782296 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-ae0f337f-7b85-4f7e-b84a-870d369f2b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:03.861020 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1175/2368] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:37:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:37:03.865630 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff263b-3e66-4d22-94a0-53f7471db14b admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:03.936737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff263b-3e66-4d22-94a0-53f7471db14b admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:03.941602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c547f808-9db7-4e49-8459-1f9a4cf82312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:03.942222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547f808-9db7-4e49-8459-1f9a4cf82312 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:03.942533 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547f808-9db7-4e49-8459-1f9a4cf82312 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:03.942809 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547f808-9db7-4e49-8459-1f9a4cf82312 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:03.943451 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c547f808-9db7-4e49-8459-1f9a4cf82312 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:03.945365 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c547f808-9db7-4e49-8459-1f9a4cf82312 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:04.031761 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1194/2369] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:37:03 2022] POST /identity/v3/projects => generated 383 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:37:04.035928 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f337f-7b85-4f7e-b84a-870d369f2b93 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:04.047832 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f337f-7b85-4f7e-b84a-870d369f2b93 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:04.067441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:04.117187 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c700905-04dc-4144-bdf2-f93f260bd36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:04.117812 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c700905-04dc-4144-bdf2-f93f260bd36c admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:04.118086 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c700905-04dc-4144-bdf2-f93f260bd36c admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:04.118355 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c700905-04dc-4144-bdf2-f93f260bd36c admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:04.119028 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c700905-04dc-4144-bdf2-f93f260bd36c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:04.121020 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c700905-04dc-4144-bdf2-f93f260bd36c admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:04.220284 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1176/2370] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:37:04 2022] POST /identity/v3/users => generated 298 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:37:04.224378 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c547f808-9db7-4e49-8459-1f9a4cf82312 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:04.235843 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c547f808-9db7-4e49-8459-1f9a4cf82312 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:04.240810 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282b326-bded-4d3f-adac-1c076f9596e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:04.241408 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282b326-bded-4d3f-adac-1c076f9596e7 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:04.241685 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282b326-bded-4d3f-adac-1c076f9596e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:04.241953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282b326-bded-4d3f-adac-1c076f9596e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:04.242670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1282b326-bded-4d3f-adac-1c076f9596e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:04.244269 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1282b326-bded-4d3f-adac-1c076f9596e7 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:04.385922 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1195/2371] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:37:04.390942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c700905-04dc-4144-bdf2-f93f260bd36c admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:04.401419 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c700905-04dc-4144-bdf2-f93f260bd36c admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:04.406237 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c694b8a-1d3c-4c4f-836f-8763428ef522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:04.406953 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c694b8a-1d3c-4c4f-836f-8763428ef522 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:04.407227 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c694b8a-1d3c-4c4f-836f-8763428ef522 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:04.407495 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c694b8a-1d3c-4c4f-836f-8763428ef522 admin admin] PATH_INFO: `/v3/projects/3ff5c984f6264d2fb023e01e80b520d4/users/95ae0411267f47718e44a2c0cc4b2d9d/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:04.423743 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c694b8a-1d3c-4c4f-836f-8763428ef522 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ff5c984f6264d2fb023e01e80b520d4, user_id=95ae0411267f47718e44a2c0cc4b2d9d, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:04.426439 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c694b8a-1d3c-4c4f-836f-8763428ef522 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:04.512114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1177/2372] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:37:04 2022] PUT /identity/v3/projects/3ff5c984f6264d2fb023e01e80b520d4/users/95ae0411267f47718e44a2c0cc4b2d9d/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:37:04.517150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282b326-bded-4d3f-adac-1c076f9596e7 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:04.527974 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282b326-bded-4d3f-adac-1c076f9596e7 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:04.589327 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9aace-75a7-44a6-94ce-97bca88be038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:04.589947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9aace-75a7-44a6-94ce-97bca88be038 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:04.590251 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9aace-75a7-44a6-94ce-97bca88be038 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:04.590551 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9aace-75a7-44a6-94ce-97bca88be038 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:04.591292 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9aace-75a7-44a6-94ce-97bca88be038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:04.592906 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9aace-75a7-44a6-94ce-97bca88be038 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:04.603729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1196/2373] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:37:04.609000 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c694b8a-1d3c-4c4f-836f-8763428ef522 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:04.620104 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c694b8a-1d3c-4c4f-836f-8763428ef522 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:04.625753 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048840b2-d9c5-4d97-86fe-48e73eb7225e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:04.626495 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048840b2-d9c5-4d97-86fe-48e73eb7225e admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:04.626771 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048840b2-d9c5-4d97-86fe-48e73eb7225e admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:04.627045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048840b2-d9c5-4d97-86fe-48e73eb7225e admin admin] PATH_INFO: `/v3/projects/3ff5c984f6264d2fb023e01e80b520d4/users/95ae0411267f47718e44a2c0cc4b2d9d/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:04.629022 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048840b2-d9c5-4d97-86fe-48e73eb7225e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ff5c984f6264d2fb023e01e80b520d4, user_id=95ae0411267f47718e44a2c0cc4b2d9d, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:04.631754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048840b2-d9c5-4d97-86fe-48e73eb7225e admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:04.644190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1178/2374] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:37:04 2022] PUT /identity/v3/projects/3ff5c984f6264d2fb023e01e80b520d4/users/95ae0411267f47718e44a2c0cc4b2d9d/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:37:04.715667 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2d52d-a0a6-43bd-86d1-39f97f8a1b52 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:04.715944 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2d52d-a0a6-43bd-86d1-39f97f8a1b52 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:04.716198 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2d52d-a0a6-43bd-86d1-39f97f8a1b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:04.746053 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-c2b2d52d-a0a6-43bd-86d1-39f97f8a1b52 None None] MFA Rules not processed for user `95ae0411267f47718e44a2c0cc4b2d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:37:04.747900 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-c2b2d52d-a0a6-43bd-86d1-39f97f8a1b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:04.756553 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:04.824896 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1197/2375] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:37:04 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:37:14.639926 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=36,keystone:INSERT=5 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:14.837893 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=25 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:27.592359 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048840b2-d9c5-4d97-86fe-48e73eb7225e admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:27.594185 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-048840b2-d9c5-4d97-86fe-48e73eb7225e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:27.657405 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048840b2-d9c5-4d97-86fe-48e73eb7225e admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:27.657405 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af026f91-f80c-41f7-9bf7-eecb76e2be06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:27.657976 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af026f91-f80c-41f7-9bf7-eecb76e2be06 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:27.658244 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af026f91-f80c-41f7-9bf7-eecb76e2be06 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:27.658541 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af026f91-f80c-41f7-9bf7-eecb76e2be06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:27.660260 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-af026f91-f80c-41f7-9bf7-eecb76e2be06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:27.719385 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af026f91-f80c-41f7-9bf7-eecb76e2be06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:27.720829 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af026f91-f80c-41f7-9bf7-eecb76e2be06 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:27.729690 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1179/2376] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:37:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:37:27.773619 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2d52d-a0a6-43bd-86d1-39f97f8a1b52 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:27.783589 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2d52d-a0a6-43bd-86d1-39f97f8a1b52 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:27.787150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb5655-22ad-4ad6-90a4-2749b47f96ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:27.787737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb5655-22ad-4ad6-90a4-2749b47f96ce service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:27.788009 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb5655-22ad-4ad6-90a4-2749b47f96ce service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:27.788276 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb5655-22ad-4ad6-90a4-2749b47f96ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:27.789978 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-20cb5655-22ad-4ad6-90a4-2749b47f96ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:27.863300 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb5655-22ad-4ad6-90a4-2749b47f96ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:27.864746 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb5655-22ad-4ad6-90a4-2749b47f96ce service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:27.876449 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1198/2377] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:37:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:37:28.564130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af026f91-f80c-41f7-9bf7-eecb76e2be06 service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:28.572664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af026f91-f80c-41f7-9bf7-eecb76e2be06 service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:28.576943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f2048-9575-42c0-b888-079e7113cb0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:28.577541 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f2048-9575-42c0-b888-079e7113cb0f service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:28.585283 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f2048-9575-42c0-b888-079e7113cb0f service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:28.585283 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f2048-9575-42c0-b888-079e7113cb0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:28.585283 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-ef9f2048-9575-42c0-b888-079e7113cb0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:37:28.646854 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f2048-9575-42c0-b888-079e7113cb0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:28.648353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f2048-9575-42c0-b888-079e7113cb0f service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:28.659327 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1180/2378] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:37:28 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:37:29.032107 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb5655-22ad-4ad6-90a4-2749b47f96ce service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:37:29.040655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb5655-22ad-4ad6-90a4-2749b47f96ce service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:37:29.045552 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5a777-55a2-4c41-96c1-d780a1973488 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:37:29.046153 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5a777-55a2-4c41-96c1-d780a1973488 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:37:29.046467 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5a777-55a2-4c41-96c1-d780a1973488 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:37:29.053263 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5a777-55a2-4c41-96c1-d780a1973488 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:37:29.056353 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5a777-55a2-4c41-96c1-d780a1973488 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:37:29.057778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5a777-55a2-4c41-96c1-d780a1973488 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:37:29.067649 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1199/2379] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:37:29 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:37:38.654563 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=78 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:37:39.063262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=40 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:23.308516 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f2048-9575-42c0-b888-079e7113cb0f service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:23.308956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-ef9f2048-9575-42c0-b888-079e7113cb0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:23.323669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f2048-9575-42c0-b888-079e7113cb0f service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:23.328038 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891c824-f598-4168-9c8a-fdb7e681e3ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:23.328667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891c824-f598-4168-9c8a-fdb7e681e3ba service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:23.331581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891c824-f598-4168-9c8a-fdb7e681e3ba service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:23.331581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891c824-f598-4168-9c8a-fdb7e681e3ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:23.331581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a891c824-f598-4168-9c8a-fdb7e681e3ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:23.373183 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5a777-55a2-4c41-96c1-d780a1973488 service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:23.375137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-06f5a777-55a2-4c41-96c1-d780a1973488 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:23.444054 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a891c824-f598-4168-9c8a-fdb7e681e3ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:23.445503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a891c824-f598-4168-9c8a-fdb7e681e3ba service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:23.458658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1181/2380] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:39:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:39:23.459491 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891c824-f598-4168-9c8a-fdb7e681e3ba service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:23.463689 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5a777-55a2-4c41-96c1-d780a1973488 service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:23.468186 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891c824-f598-4168-9c8a-fdb7e681e3ba service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:23.469170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a4ee0-f58b-49c4-8991-5c8e23f1ba3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:23.470616 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a4ee0-f58b-49c4-8991-5c8e23f1ba3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:23.473603 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a4ee0-f58b-49c4-8991-5c8e23f1ba3b admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:23.473603 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a4ee0-f58b-49c4-8991-5c8e23f1ba3b admin admin] PATH_INFO: `/v3/users/24bbc03d3c80452ebe5cc63e1014b5ea` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:23.473603 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a4ee0-f58b-49c4-8991-5c8e23f1ba3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=24bbc03d3c80452ebe5cc63e1014b5ea)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:23.473823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0941b6-d60e-4372-952a-dcb3cb88563f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:23.473823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0941b6-d60e-4372-952a-dcb3cb88563f service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:23.473823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0941b6-d60e-4372-952a-dcb3cb88563f service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:23.474125 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0941b6-d60e-4372-952a-dcb3cb88563f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:23.474743 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a4ee0-f58b-49c4-8991-5c8e23f1ba3b admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:23.484533 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0941b6-d60e-4372-952a-dcb3cb88563f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:23.485990 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0941b6-d60e-4372-952a-dcb3cb88563f service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:23.498182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1182/2381] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:39:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:39:23.623158 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0941b6-d60e-4372-952a-dcb3cb88563f service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:23.633916 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0941b6-d60e-4372-952a-dcb3cb88563f service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:23.641605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c4197-b9ea-44bc-913b-491ae10e5b08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:23.642217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c4197-b9ea-44bc-913b-491ae10e5b08 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:23.642546 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c4197-b9ea-44bc-913b-491ae10e5b08 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:23.642820 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c4197-b9ea-44bc-913b-491ae10e5b08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:23.702725 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c35a4ee0-f58b-49c4-8991-5c8e23f1ba3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24bbc03d3c80452ebe5cc63e1014b5ea'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:23.713710 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c4197-b9ea-44bc-913b-491ae10e5b08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:23.715289 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c4197-b9ea-44bc-913b-491ae10e5b08 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:23.718511 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c35a4ee0-f58b-49c4-8991-5c8e23f1ba3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24bbc03d3c80452ebe5cc63e1014b5ea'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:23.725211 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1183/2382] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:39:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:39:23.726154 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c35a4ee0-f58b-49c4-8991-5c8e23f1ba3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24bbc03d3c80452ebe5cc63e1014b5ea'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:23.848377 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c35a4ee0-f58b-49c4-8991-5c8e23f1ba3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24bbc03d3c80452ebe5cc63e1014b5ea'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:23.849782 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c35a4ee0-f58b-49c4-8991-5c8e23f1ba3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24bbc03d3c80452ebe5cc63e1014b5ea'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:23.859908 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1200/2383] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:39:23 2022] DELETE /identity/v3/users/24bbc03d3c80452ebe5cc63e1014b5ea => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:23.864355 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad690c5-0dd5-4430-b1f3-06bdef4be74e None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:23.864682 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad690c5-0dd5-4430-b1f3-06bdef4be74e None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:23.864966 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad690c5-0dd5-4430-b1f3-06bdef4be74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:23.888466 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-9ad690c5-0dd5-4430-b1f3-06bdef4be74e None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:23.889917 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-9ad690c5-0dd5-4430-b1f3-06bdef4be74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:23.964512 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1184/2384] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:39:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:39:23.978820 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a4ee0-f58b-49c4-8991-5c8e23f1ba3b admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:23.980247 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-c35a4ee0-f58b-49c4-8991-5c8e23f1ba3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:23.996511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad690c5-0dd5-4430-b1f3-06bdef4be74e None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:23.997912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-9ad690c5-0dd5-4430-b1f3-06bdef4be74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:24.015295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:24.045187 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a4ee0-f58b-49c4-8991-5c8e23f1ba3b admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:24.049044 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae328ace-f0c3-4816-b0ce-90841542b790 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:24.049647 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae328ace-f0c3-4816-b0ce-90841542b790 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:24.049923 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae328ace-f0c3-4816-b0ce-90841542b790 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:24.050193 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae328ace-f0c3-4816-b0ce-90841542b790 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:24.061361 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae328ace-f0c3-4816-b0ce-90841542b790 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:24.062823 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae328ace-f0c3-4816-b0ce-90841542b790 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:24.063090 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad690c5-0dd5-4430-b1f3-06bdef4be74e None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:24.066772 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8893b-4e68-445e-9af4-d57b32a5c3fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:24.067383 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8893b-4e68-445e-9af4-d57b32a5c3fe service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:24.067656 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8893b-4e68-445e-9af4-d57b32a5c3fe service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:24.067932 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8893b-4e68-445e-9af4-d57b32a5c3fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:24.069682 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-62e8893b-4e68-445e-9af4-d57b32a5c3fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:24.073392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1201/2385] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:39:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:39:24.153356 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8893b-4e68-445e-9af4-d57b32a5c3fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:24.154810 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8893b-4e68-445e-9af4-d57b32a5c3fe service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:24.167046 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1185/2386] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:39:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:39:24.498652 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae328ace-f0c3-4816-b0ce-90841542b790 service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:24.500089 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-ae328ace-f0c3-4816-b0ce-90841542b790 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:24.512938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae328ace-f0c3-4816-b0ce-90841542b790 service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:24.517475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706dd15-a2e5-4c55-8baa-4ed33b1c2b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:24.518525 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706dd15-a2e5-4c55-8baa-4ed33b1c2b93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:24.518525 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706dd15-a2e5-4c55-8baa-4ed33b1c2b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:24.518754 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706dd15-a2e5-4c55-8baa-4ed33b1c2b93 admin admin] PATH_INFO: `/v3/users/e38560c102ef446c95892d546a2450af` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:24.519998 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5706dd15-a2e5-4c55-8baa-4ed33b1c2b93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e38560c102ef446c95892d546a2450af)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:24.521996 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5706dd15-a2e5-4c55-8baa-4ed33b1c2b93 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:24.538649 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:24.558824 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8893b-4e68-445e-9af4-d57b32a5c3fe service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:24.561948 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-62e8893b-4e68-445e-9af4-d57b32a5c3fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:24.581937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8893b-4e68-445e-9af4-d57b32a5c3fe service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:24.589829 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fa4cb-35d3-410b-8f25-f1efff7e1261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:24.590530 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fa4cb-35d3-410b-8f25-f1efff7e1261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:24.590806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fa4cb-35d3-410b-8f25-f1efff7e1261 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:24.591072 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fa4cb-35d3-410b-8f25-f1efff7e1261 admin admin] PATH_INFO: `/v3/projects/a368e821847243caa3e8c0cc34fc4769` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:24.592409 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73fa4cb-35d3-410b-8f25-f1efff7e1261 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a368e821847243caa3e8c0cc34fc4769)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:24.594490 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73fa4cb-35d3-410b-8f25-f1efff7e1261 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:24.773958 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-5706dd15-a2e5-4c55-8baa-4ed33b1c2b93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e38560c102ef446c95892d546a2450af'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:24.781972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-f73fa4cb-35d3-410b-8f25-f1efff7e1261 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a368e821847243caa3e8c0cc34fc4769', 'request_id': 'req-f73fa4cb-35d3-410b-8f25-f1efff7e1261'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:24.788692 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-f73fa4cb-35d3-410b-8f25-f1efff7e1261 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a368e821847243caa3e8c0cc34fc4769', 'request_id': 'req-f73fa4cb-35d3-410b-8f25-f1efff7e1261'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:24.790280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-5706dd15-a2e5-4c55-8baa-4ed33b1c2b93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e38560c102ef446c95892d546a2450af'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:24.796420 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-f73fa4cb-35d3-410b-8f25-f1efff7e1261 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a368e821847243caa3e8c0cc34fc4769', 'request_id': 'req-f73fa4cb-35d3-410b-8f25-f1efff7e1261'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:24.799806 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-5706dd15-a2e5-4c55-8baa-4ed33b1c2b93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e38560c102ef446c95892d546a2450af'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:24.939353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-f73fa4cb-35d3-410b-8f25-f1efff7e1261 admin admin] The token cache is being invalidate because project a368e821847243caa3e8c0cc34fc4769 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:39:24.939644 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-f73fa4cb-35d3-410b-8f25-f1efff7e1261 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:24.940331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-f73fa4cb-35d3-410b-8f25-f1efff7e1261 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:24.941842 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1186/2387] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:39:24 2022] DELETE /identity/v3/projects/a368e821847243caa3e8c0cc34fc4769 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:24.948115 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fa4cb-35d3-410b-8f25-f1efff7e1261 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:24.949567 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-f73fa4cb-35d3-410b-8f25-f1efff7e1261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:25.006669 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:25.038147 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fa4cb-35d3-410b-8f25-f1efff7e1261 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:25.038836 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-5706dd15-a2e5-4c55-8baa-4ed33b1c2b93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e38560c102ef446c95892d546a2450af'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:25.039608 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-5706dd15-a2e5-4c55-8baa-4ed33b1c2b93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e38560c102ef446c95892d546a2450af'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:25.044517 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb4dddb-b12a-48f8-a378-fe7256644ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:25.045871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4dddb-b12a-48f8-a378-fe7256644ebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:25.046239 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4dddb-b12a-48f8-a378-fe7256644ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:25.046643 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4dddb-b12a-48f8-a378-fe7256644ebf admin admin] PATH_INFO: `/v3/users/d8f01b1961d24515915fa16f295e4899` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:25.048081 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb4dddb-b12a-48f8-a378-fe7256644ebf admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8f01b1961d24515915fa16f295e4899)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:25.050591 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb4dddb-b12a-48f8-a378-fe7256644ebf admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:25.054285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1202/2388] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:39:24 2022] DELETE /identity/v3/users/e38560c102ef446c95892d546a2450af => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:25.059991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0918173-cdd8-4f4e-b62c-a1b18e6cb613 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:25.060520 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0918173-cdd8-4f4e-b62c-a1b18e6cb613 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:25.061098 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0918173-cdd8-4f4e-b62c-a1b18e6cb613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:25.108108 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-f0918173-cdd8-4f4e-b62c-a1b18e6cb613 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:25.109330 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-f0918173-cdd8-4f4e-b62c-a1b18e6cb613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:25.176057 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1203/2389] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:39:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:39:25.186403 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0918173-cdd8-4f4e-b62c-a1b18e6cb613 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:25.187835 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-f0918173-cdd8-4f4e-b62c-a1b18e6cb613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:25.255440 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0918173-cdd8-4f4e-b62c-a1b18e6cb613 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:25.264605 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd6bea6-5b7e-4be9-8686-8ed36db4ddbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:25.265226 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd6bea6-5b7e-4be9-8686-8ed36db4ddbd service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:25.265506 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd6bea6-5b7e-4be9-8686-8ed36db4ddbd service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:25.265775 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd6bea6-5b7e-4be9-8686-8ed36db4ddbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:25.277707 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd6bea6-5b7e-4be9-8686-8ed36db4ddbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:25.282427 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd6bea6-5b7e-4be9-8686-8ed36db4ddbd service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:25.301679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-4fb4dddb-b12a-48f8-a378-fe7256644ebf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8f01b1961d24515915fa16f295e4899'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:25.302432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1204/2390] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:39:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:39:25.368138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-4fb4dddb-b12a-48f8-a378-fe7256644ebf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8f01b1961d24515915fa16f295e4899'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:25.369846 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-4fb4dddb-b12a-48f8-a378-fe7256644ebf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8f01b1961d24515915fa16f295e4899'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:25.370611 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-4fb4dddb-b12a-48f8-a378-fe7256644ebf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8f01b1961d24515915fa16f295e4899'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:25.385903 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-4fb4dddb-b12a-48f8-a378-fe7256644ebf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8f01b1961d24515915fa16f295e4899'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:25.399469 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1187/2391] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:39:24 2022] DELETE /identity/v3/users/d8f01b1961d24515915fa16f295e4899 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:39:25.659834 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd6bea6-5b7e-4be9-8686-8ed36db4ddbd service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:25.661333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-7fd6bea6-5b7e-4be9-8686-8ed36db4ddbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:25.722987 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:25.760535 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd6bea6-5b7e-4be9-8686-8ed36db4ddbd service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:25.849817 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b87b0-76d3-49b9-942a-5db5dc141a97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:25.850771 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b87b0-76d3-49b9-942a-5db5dc141a97 service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:25.853396 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b87b0-76d3-49b9-942a-5db5dc141a97 service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:25.853806 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b87b0-76d3-49b9-942a-5db5dc141a97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:25.856137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-9b6b87b0-76d3-49b9-942a-5db5dc141a97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:25.872464 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6b87b0-76d3-49b9-942a-5db5dc141a97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:25.874367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6b87b0-76d3-49b9-942a-5db5dc141a97 service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:25.897181 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1205/2392] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:39:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:39:26.142005 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb4dddb-b12a-48f8-a378-fe7256644ebf admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:26.143615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-4fb4dddb-b12a-48f8-a378-fe7256644ebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:26.171389 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b87b0-76d3-49b9-942a-5db5dc141a97 service nova] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:26.172890 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-9b6b87b0-76d3-49b9-942a-5db5dc141a97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:26.232812 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb4dddb-b12a-48f8-a378-fe7256644ebf admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:26.237829 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80b33f-a390-4daa-9e40-2bab878d7b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:26.238517 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80b33f-a390-4daa-9e40-2bab878d7b7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:26.238796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80b33f-a390-4daa-9e40-2bab878d7b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:26.239064 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80b33f-a390-4daa-9e40-2bab878d7b7c admin admin] PATH_INFO: `/v3/projects/94607d6a3c184e5980dfd87f0d5462cf` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:26.240340 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b80b33f-a390-4daa-9e40-2bab878d7b7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=94607d6a3c184e5980dfd87f0d5462cf)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:26.242583 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b80b33f-a390-4daa-9e40-2bab878d7b7c admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:26.260587 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b87b0-76d3-49b9-942a-5db5dc141a97 service nova] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:26.265675 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cddebc4-78b1-4a9b-9583-70cc0f3bcbf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:26.265986 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cddebc4-78b1-4a9b-9583-70cc0f3bcbf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:26.266324 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cddebc4-78b1-4a9b-9583-70cc0f3bcbf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:26.266684 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cddebc4-78b1-4a9b-9583-70cc0f3bcbf6 admin admin] PATH_INFO: `/v3/projects/e829e37fbc0d4726bcf3030d2fe5cc22` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:26.267989 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cddebc4-78b1-4a9b-9583-70cc0f3bcbf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e829e37fbc0d4726bcf3030d2fe5cc22)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:26.269948 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cddebc4-78b1-4a9b-9583-70cc0f3bcbf6 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:26.376954 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-0b80b33f-a390-4daa-9e40-2bab878d7b7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94607d6a3c184e5980dfd87f0d5462cf', 'request_id': 'req-0b80b33f-a390-4daa-9e40-2bab878d7b7c'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:26.385137 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-0b80b33f-a390-4daa-9e40-2bab878d7b7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94607d6a3c184e5980dfd87f0d5462cf', 'request_id': 'req-0b80b33f-a390-4daa-9e40-2bab878d7b7c'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:26.390269 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:26.392877 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-0b80b33f-a390-4daa-9e40-2bab878d7b7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94607d6a3c184e5980dfd87f0d5462cf', 'request_id': 'req-0b80b33f-a390-4daa-9e40-2bab878d7b7c'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:26.414590 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-0cddebc4-78b1-4a9b-9583-70cc0f3bcbf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e829e37fbc0d4726bcf3030d2fe5cc22', 'request_id': 'req-0cddebc4-78b1-4a9b-9583-70cc0f3bcbf6'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:26.544424 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-0b80b33f-a390-4daa-9e40-2bab878d7b7c admin admin] The token cache is being invalidate because project 94607d6a3c184e5980dfd87f0d5462cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:39:26.544715 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-0b80b33f-a390-4daa-9e40-2bab878d7b7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:26.545334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-0b80b33f-a390-4daa-9e40-2bab878d7b7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:26.546877 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1188/2393] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:39:26 2022] DELETE /identity/v3/projects/94607d6a3c184e5980dfd87f0d5462cf => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:26.547686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80b33f-a390-4daa-9e40-2bab878d7b7c admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:26.548967 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-0b80b33f-a390-4daa-9e40-2bab878d7b7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:26.606155 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80b33f-a390-4daa-9e40-2bab878d7b7c admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:26.653294 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116f096-2d82-40ba-b8d1-9c3719eac067 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:26.653816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116f096-2d82-40ba-b8d1-9c3719eac067 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:26.654097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116f096-2d82-40ba-b8d1-9c3719eac067 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:26.654362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116f096-2d82-40ba-b8d1-9c3719eac067 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:26.656234 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-b116f096-2d82-40ba-b8d1-9c3719eac067 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:26.682742 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-0cddebc4-78b1-4a9b-9583-70cc0f3bcbf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e829e37fbc0d4726bcf3030d2fe5cc22', 'request_id': 'req-0cddebc4-78b1-4a9b-9583-70cc0f3bcbf6'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:26.688288 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:26.689633 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-0cddebc4-78b1-4a9b-9583-70cc0f3bcbf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e829e37fbc0d4726bcf3030d2fe5cc22', 'request_id': 'req-0cddebc4-78b1-4a9b-9583-70cc0f3bcbf6'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:26.701661 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-0cddebc4-78b1-4a9b-9583-70cc0f3bcbf6 admin admin] The token cache is being invalidate because project e829e37fbc0d4726bcf3030d2fe5cc22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:39:26.701951 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-0cddebc4-78b1-4a9b-9583-70cc0f3bcbf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:26.702653 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-0cddebc4-78b1-4a9b-9583-70cc0f3bcbf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:26.704568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1206/2394] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:39:26 2022] DELETE /identity/v3/projects/e829e37fbc0d4726bcf3030d2fe5cc22 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:39:26.709868 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cddebc4-78b1-4a9b-9583-70cc0f3bcbf6 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:26.711272 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-0cddebc4-78b1-4a9b-9583-70cc0f3bcbf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:26.723716 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b116f096-2d82-40ba-b8d1-9c3719eac067 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:26.725307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b116f096-2d82-40ba-b8d1-9c3719eac067 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:26.732429 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cddebc4-78b1-4a9b-9583-70cc0f3bcbf6 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:26.737132 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1189/2395] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:39:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:39:26.738147 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625cbe3-ce7c-42a6-b558-1feecac7d16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:26.739957 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625cbe3-ce7c-42a6-b558-1feecac7d16c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:26.739957 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625cbe3-ce7c-42a6-b558-1feecac7d16c admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:26.740289 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514365d-541b-457e-bc59-557918957d98 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:26.740368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625cbe3-ce7c-42a6-b558-1feecac7d16c admin admin] PATH_INFO: `/v3/users/7221971389bb45b7815f0a867dcc7ef4` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:26.740444 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514365d-541b-457e-bc59-557918957d98 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:26.740444 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514365d-541b-457e-bc59-557918957d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:26.741403 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625cbe3-ce7c-42a6-b558-1feecac7d16c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7221971389bb45b7815f0a867dcc7ef4)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:26.744188 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625cbe3-ce7c-42a6-b558-1feecac7d16c admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:26.761551 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-e514365d-541b-457e-bc59-557918957d98 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:26.774507 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-e514365d-541b-457e-bc59-557918957d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:26.784731 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1190/2396] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:39:26.793267 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecdaae-4d62-450d-98a4-ab3699021095 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:26.793716 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecdaae-4d62-450d-98a4-ab3699021095 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:26.794054 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecdaae-4d62-450d-98a4-ab3699021095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:26.839423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-03ecdaae-4d62-450d-98a4-ab3699021095 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:26.840686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-03ecdaae-4d62-450d-98a4-ab3699021095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:26.866922 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1191/2397] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:39:26.867958 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ecdaae-4d62-450d-98a4-ab3699021095 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:26.879497 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ecdaae-4d62-450d-98a4-ab3699021095 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:26.885749 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd63b56b-78d1-49fb-a663-1111335716a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:26.887496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63b56b-78d1-49fb-a663-1111335716a1 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:26.887496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63b56b-78d1-49fb-a663-1111335716a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:26.887496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63b56b-78d1-49fb-a663-1111335716a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:26.888054 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd63b56b-78d1-49fb-a663-1111335716a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:39:26.888423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd63b56b-78d1-49fb-a663-1111335716a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:26.890480 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd63b56b-78d1-49fb-a663-1111335716a1 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:26.917278 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1192/2398] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:39:26.921900 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd63b56b-78d1-49fb-a663-1111335716a1 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:26.933222 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd63b56b-78d1-49fb-a663-1111335716a1 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:26.939009 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2dadb-4b89-4956-93ce-90d7513d6bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:26.939828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2dadb-4b89-4956-93ce-90d7513d6bd7 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:26.940593 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2dadb-4b89-4956-93ce-90d7513d6bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:26.940868 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2dadb-4b89-4956-93ce-90d7513d6bd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:26.941514 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc2dadb-4b89-4956-93ce-90d7513d6bd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:26.943721 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc2dadb-4b89-4956-93ce-90d7513d6bd7 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:26.975101 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-8625cbe3-ce7c-42a6-b558-1feecac7d16c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7221971389bb45b7815f0a867dcc7ef4'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:26.995193 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-8625cbe3-ce7c-42a6-b558-1feecac7d16c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7221971389bb45b7815f0a867dcc7ef4'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:27.004653 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-8625cbe3-ce7c-42a6-b558-1feecac7d16c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7221971389bb45b7815f0a867dcc7ef4'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:27.053302 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1193/2399] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:39:26 2022] POST /identity/v3/projects => generated 381 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:39:27.063885 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2dadb-4b89-4956-93ce-90d7513d6bd7 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:27.068537 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:27.086286 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-8625cbe3-ce7c-42a6-b558-1feecac7d16c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7221971389bb45b7815f0a867dcc7ef4'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:27.086945 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-8625cbe3-ce7c-42a6-b558-1feecac7d16c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7221971389bb45b7815f0a867dcc7ef4'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:27.095655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1207/2400] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:39:26 2022] DELETE /identity/v3/users/7221971389bb45b7815f0a867dcc7ef4 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:27.176137 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2dadb-4b89-4956-93ce-90d7513d6bd7 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:27.245071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8f895-da0a-4e44-ba6f-53379f184aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:27.245699 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8f895-da0a-4e44-ba6f-53379f184aaf admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:27.245980 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8f895-da0a-4e44-ba6f-53379f184aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:27.246259 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8f895-da0a-4e44-ba6f-53379f184aaf admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:27.247094 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d8f895-da0a-4e44-ba6f-53379f184aaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:27.250383 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d8f895-da0a-4e44-ba6f-53379f184aaf admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:27.345633 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1194/2401] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:39:27 2022] POST /identity/v3/users => generated 291 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:39:27.347056 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625cbe3-ce7c-42a6-b558-1feecac7d16c admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:27.351182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-8625cbe3-ce7c-42a6-b558-1feecac7d16c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:27.361770 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8f895-da0a-4e44-ba6f-53379f184aaf admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:27.364219 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-12d8f895-da0a-4e44-ba6f-53379f184aaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:27.373890 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625cbe3-ce7c-42a6-b558-1feecac7d16c admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:27.379171 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792756f7-3c85-488a-b9e1-24a4395598c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:27.379977 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792756f7-3c85-488a-b9e1-24a4395598c2 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:27.380156 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792756f7-3c85-488a-b9e1-24a4395598c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:27.380592 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792756f7-3c85-488a-b9e1-24a4395598c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:27.381937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792756f7-3c85-488a-b9e1-24a4395598c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:27.384693 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792756f7-3c85-488a-b9e1-24a4395598c2 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:27.400673 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1208/2402] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:39:27.402432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792756f7-3c85-488a-b9e1-24a4395598c2 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:27.423202 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792756f7-3c85-488a-b9e1-24a4395598c2 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:27.429600 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402cf3e-39c6-4845-9103-32fe5cc53ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:27.430537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402cf3e-39c6-4845-9103-32fe5cc53ada admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:27.430932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402cf3e-39c6-4845-9103-32fe5cc53ada admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:27.431163 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402cf3e-39c6-4845-9103-32fe5cc53ada admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:27.431861 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c402cf3e-39c6-4845-9103-32fe5cc53ada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:27.433662 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c402cf3e-39c6-4845-9103-32fe5cc53ada admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:27.451773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1209/2403] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:39:27.458612 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8f895-da0a-4e44-ba6f-53379f184aaf admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:27.458864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402cf3e-39c6-4845-9103-32fe5cc53ada admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:27.463810 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e49f3e-5724-487d-944e-cab9d578c1f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:27.464674 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e49f3e-5724-487d-944e-cab9d578c1f7 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:27.465053 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e49f3e-5724-487d-944e-cab9d578c1f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:27.465311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e49f3e-5724-487d-944e-cab9d578c1f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:27.467381 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-e7e49f3e-5724-487d-944e-cab9d578c1f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:27.471038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402cf3e-39c6-4845-9103-32fe5cc53ada admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:27.477162 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2dc8d-52e1-4613-8189-9a815cdc1f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:27.477970 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2dc8d-52e1-4613-8189-9a815cdc1f1a admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:27.478484 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2dc8d-52e1-4613-8189-9a815cdc1f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:27.478915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2dc8d-52e1-4613-8189-9a815cdc1f1a admin admin] PATH_INFO: `/v3/projects/44f6fc01500b4750a9b3882424813c13/users/35dae86d2bd24a3a9b27489422552d26/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:27.482310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e49f3e-5724-487d-944e-cab9d578c1f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:27.486035 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e49f3e-5724-487d-944e-cab9d578c1f7 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:27.498365 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c2dc8d-52e1-4613-8189-9a815cdc1f1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=44f6fc01500b4750a9b3882424813c13, user_id=35dae86d2bd24a3a9b27489422552d26, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:27.501721 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1195/2404] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:39:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:39:27.501918 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c2dc8d-52e1-4613-8189-9a815cdc1f1a admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:27.631956 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1210/2405] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:39:27 2022] PUT /identity/v3/projects/44f6fc01500b4750a9b3882424813c13/users/35dae86d2bd24a3a9b27489422552d26/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:27.695837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2355af-104f-4795-9a05-c6289b5b2ea1 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:27.697537 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2355af-104f-4795-9a05-c6289b5b2ea1 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:27.697994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2355af-104f-4795-9a05-c6289b5b2ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:27.739278 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-4b2355af-104f-4795-9a05-c6289b5b2ea1 None None] MFA Rules not processed for user `35dae86d2bd24a3a9b27489422552d26`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:27.740604 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-4b2355af-104f-4795-9a05-c6289b5b2ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:27.751358 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2dc8d-52e1-4613-8189-9a815cdc1f1a admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:27.753435 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-06c2dc8d-52e1-4613-8189-9a815cdc1f1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:27.837376 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:27.852693 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1196/2406] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:39:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:39:27.857948 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2355af-104f-4795-9a05-c6289b5b2ea1 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:27.908401 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2dc8d-52e1-4613-8189-9a815cdc1f1a admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:27.913882 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a2b28-ed95-4e7f-ba1a-a3fdf2a61cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:27.914562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a2b28-ed95-4e7f-ba1a-a3fdf2a61cd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:27.914844 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a2b28-ed95-4e7f-ba1a-a3fdf2a61cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:27.915120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a2b28-ed95-4e7f-ba1a-a3fdf2a61cd3 admin admin] PATH_INFO: `/v3/users/bccf1c23c68a4987882eedad6ec5c106` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:27.932294 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a2b28-ed95-4e7f-ba1a-a3fdf2a61cd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bccf1c23c68a4987882eedad6ec5c106)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:27.935381 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a2b28-ed95-4e7f-ba1a-a3fdf2a61cd3 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:27.950956 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:27.967079 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2355af-104f-4795-9a05-c6289b5b2ea1 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:27.980035 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7cb39-09ad-40d3-a7f6-c0dd78c5d8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:27.980674 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7cb39-09ad-40d3-a7f6-c0dd78c5d8be admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:27.980956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7cb39-09ad-40d3-a7f6-c0dd78c5d8be admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:27.981229 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7cb39-09ad-40d3-a7f6-c0dd78c5d8be admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:27.983745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f7cb39-09ad-40d3-a7f6-c0dd78c5d8be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:27.985780 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f7cb39-09ad-40d3-a7f6-c0dd78c5d8be admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:28.146055 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1197/2407] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:39:27 2022] POST /identity/v3/projects => generated 381 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:39:28.147203 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7cb39-09ad-40d3-a7f6-c0dd78c5d8be admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:28.149196 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-97f7cb39-09ad-40d3-a7f6-c0dd78c5d8be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:28.183524 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ab3a2b28-ed95-4e7f-ba1a-a3fdf2a61cd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bccf1c23c68a4987882eedad6ec5c106'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:28.201310 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ab3a2b28-ed95-4e7f-ba1a-a3fdf2a61cd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bccf1c23c68a4987882eedad6ec5c106'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:28.211414 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ab3a2b28-ed95-4e7f-ba1a-a3fdf2a61cd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bccf1c23c68a4987882eedad6ec5c106'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:28.223381 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7cb39-09ad-40d3-a7f6-c0dd78c5d8be admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:28.229543 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a6b21-e260-4998-817d-b1530cbfdd98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:28.230189 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a6b21-e260-4998-817d-b1530cbfdd98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:28.230724 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a6b21-e260-4998-817d-b1530cbfdd98 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:28.232564 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a6b21-e260-4998-817d-b1530cbfdd98 admin admin] PATH_INFO: `/v3/projects/d86be09318244848a615ad51cf55ca76` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:28.233761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a6b21-e260-4998-817d-b1530cbfdd98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d86be09318244848a615ad51cf55ca76)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:28.235862 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a6b21-e260-4998-817d-b1530cbfdd98 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:28.270519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ab3a2b28-ed95-4e7f-ba1a-a3fdf2a61cd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bccf1c23c68a4987882eedad6ec5c106'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:28.271133 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ab3a2b28-ed95-4e7f-ba1a-a3fdf2a61cd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bccf1c23c68a4987882eedad6ec5c106'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:28.278879 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1211/2408] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:39:27 2022] DELETE /identity/v3/users/bccf1c23c68a4987882eedad6ec5c106 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:28.282872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a2b28-ed95-4e7f-ba1a-a3fdf2a61cd3 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:28.284376 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-ab3a2b28-ed95-4e7f-ba1a-a3fdf2a61cd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:28.347061 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-d07a6b21-e260-4998-817d-b1530cbfdd98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd86be09318244848a615ad51cf55ca76', 'request_id': 'req-d07a6b21-e260-4998-817d-b1530cbfdd98'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:28.354072 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-d07a6b21-e260-4998-817d-b1530cbfdd98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd86be09318244848a615ad51cf55ca76', 'request_id': 'req-d07a6b21-e260-4998-817d-b1530cbfdd98'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:28.360929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-d07a6b21-e260-4998-817d-b1530cbfdd98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd86be09318244848a615ad51cf55ca76', 'request_id': 'req-d07a6b21-e260-4998-817d-b1530cbfdd98'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:28.374968 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a2b28-ed95-4e7f-ba1a-a3fdf2a61cd3 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:28.441015 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f705089-0bdc-4a5e-88d2-5317c9f446e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:28.441540 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f705089-0bdc-4a5e-88d2-5317c9f446e0 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:28.441809 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f705089-0bdc-4a5e-88d2-5317c9f446e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:28.442073 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f705089-0bdc-4a5e-88d2-5317c9f446e0 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:28.444341 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f705089-0bdc-4a5e-88d2-5317c9f446e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:28.455303 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f705089-0bdc-4a5e-88d2-5317c9f446e0 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:28.457842 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-d07a6b21-e260-4998-817d-b1530cbfdd98 admin admin] The token cache is being invalidate because project d86be09318244848a615ad51cf55ca76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:39:28.458603 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-d07a6b21-e260-4998-817d-b1530cbfdd98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:28.459419 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-d07a6b21-e260-4998-817d-b1530cbfdd98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:28.464715 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1198/2409] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:39:28 2022] DELETE /identity/v3/projects/d86be09318244848a615ad51cf55ca76 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:28.466378 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6321e9-d1e6-4103-a539-157fbd87c01c None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:28.466976 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6321e9-d1e6-4103-a539-157fbd87c01c None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:28.467411 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6321e9-d1e6-4103-a539-157fbd87c01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:28.475229 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:28.495586 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-7f6321e9-d1e6-4103-a539-157fbd87c01c None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:28.496826 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-7f6321e9-d1e6-4103-a539-157fbd87c01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:28.504882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1199/2410] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:39:28.505740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6321e9-d1e6-4103-a539-157fbd87c01c None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:28.506975 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-7f6321e9-d1e6-4103-a539-157fbd87c01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:28.519436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6321e9-d1e6-4103-a539-157fbd87c01c None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:28.524083 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9dd132-fdb3-47ab-9a15-62fd611b22a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:28.524893 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9dd132-fdb3-47ab-9a15-62fd611b22a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:28.525002 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9dd132-fdb3-47ab-9a15-62fd611b22a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:28.525282 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9dd132-fdb3-47ab-9a15-62fd611b22a7 admin admin] PATH_INFO: `/v3/users/b3d6118f7e3a4158bb456a07368ec961` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:28.526460 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9dd132-fdb3-47ab-9a15-62fd611b22a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3d6118f7e3a4158bb456a07368ec961)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:28.528364 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9dd132-fdb3-47ab-9a15-62fd611b22a7 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:28.601238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1212/2411] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:39:28 2022] POST /identity/v3/users => generated 297 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:39:28.601238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8c3e1-de84-4a7d-a66f-409aadc5c434 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:28.601238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8c3e1-de84-4a7d-a66f-409aadc5c434 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:28.601238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8c3e1-de84-4a7d-a66f-409aadc5c434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:28.625076 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-60a8c3e1-de84-4a7d-a66f-409aadc5c434 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:28.626280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-60a8c3e1-de84-4a7d-a66f-409aadc5c434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:28.635353 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1213/2412] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:39:28.636221 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8c3e1-de84-4a7d-a66f-409aadc5c434 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:28.638614 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-60a8c3e1-de84-4a7d-a66f-409aadc5c434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:28.652406 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8c3e1-de84-4a7d-a66f-409aadc5c434 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:28.657110 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51740703-f6d1-4e45-8c58-1c9633924106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:28.657763 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51740703-f6d1-4e45-8c58-1c9633924106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:28.658034 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51740703-f6d1-4e45-8c58-1c9633924106 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:28.658303 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51740703-f6d1-4e45-8c58-1c9633924106 admin admin] PATH_INFO: `/v3/projects/b9e701ae69b244d2993f92b44437a716` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:28.659435 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51740703-f6d1-4e45-8c58-1c9633924106 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9e701ae69b244d2993f92b44437a716)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:28.661314 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51740703-f6d1-4e45-8c58-1c9633924106 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:28.757910 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2d9dd132-fdb3-47ab-9a15-62fd611b22a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3d6118f7e3a4158bb456a07368ec961'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:28.771171 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-51740703-f6d1-4e45-8c58-1c9633924106 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9e701ae69b244d2993f92b44437a716', 'request_id': 'req-51740703-f6d1-4e45-8c58-1c9633924106'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:28.838731 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2d9dd132-fdb3-47ab-9a15-62fd611b22a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3d6118f7e3a4158bb456a07368ec961'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:28.839891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2d9dd132-fdb3-47ab-9a15-62fd611b22a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3d6118f7e3a4158bb456a07368ec961'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:28.840756 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2d9dd132-fdb3-47ab-9a15-62fd611b22a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3d6118f7e3a4158bb456a07368ec961'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:28.855284 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2d9dd132-fdb3-47ab-9a15-62fd611b22a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3d6118f7e3a4158bb456a07368ec961'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:28.877141 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1200/2413] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:39:28 2022] DELETE /identity/v3/users/b3d6118f7e3a4158bb456a07368ec961 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:28.877141 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9dd132-fdb3-47ab-9a15-62fd611b22a7 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:28.877141 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-2d9dd132-fdb3-47ab-9a15-62fd611b22a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:28.882751 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-51740703-f6d1-4e45-8c58-1c9633924106 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9e701ae69b244d2993f92b44437a716', 'request_id': 'req-51740703-f6d1-4e45-8c58-1c9633924106'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:28.893334 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-51740703-f6d1-4e45-8c58-1c9633924106 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9e701ae69b244d2993f92b44437a716', 'request_id': 'req-51740703-f6d1-4e45-8c58-1c9633924106'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:28.906283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:28.908012 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-51740703-f6d1-4e45-8c58-1c9633924106 admin admin] The token cache is being invalidate because project b9e701ae69b244d2993f92b44437a716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:39:28.908380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-51740703-f6d1-4e45-8c58-1c9633924106 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:28.909149 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-51740703-f6d1-4e45-8c58-1c9633924106 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:28.910946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1214/2414] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:39:28 2022] DELETE /identity/v3/projects/b9e701ae69b244d2993f92b44437a716 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:28.911686 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51740703-f6d1-4e45-8c58-1c9633924106 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:28.913675 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-51740703-f6d1-4e45-8c58-1c9633924106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:28.969623 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9dd132-fdb3-47ab-9a15-62fd611b22a7 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:28.975276 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39146ef0-8a64-4482-b42a-3853244e33b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:28.976114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39146ef0-8a64-4482-b42a-3853244e33b9 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:28.976534 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39146ef0-8a64-4482-b42a-3853244e33b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:28.976931 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39146ef0-8a64-4482-b42a-3853244e33b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:28.977703 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39146ef0-8a64-4482-b42a-3853244e33b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:28.980353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39146ef0-8a64-4482-b42a-3853244e33b9 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:28.985073 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51740703-f6d1-4e45-8c58-1c9633924106 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:28.990134 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2f5bf-cbd2-42dd-bac1-865cf85b12dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:28.990770 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2f5bf-cbd2-42dd-bac1-865cf85b12dc admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:28.991050 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2f5bf-cbd2-42dd-bac1-865cf85b12dc admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:28.991317 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2f5bf-cbd2-42dd-bac1-865cf85b12dc admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:28.991964 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de2f5bf-cbd2-42dd-bac1-865cf85b12dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:39:28.992266 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de2f5bf-cbd2-42dd-bac1-865cf85b12dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:28.993251 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1201/2415] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:39:28.994481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de2f5bf-cbd2-42dd-bac1-865cf85b12dc admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:28.997750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb3ad4-896c-4377-afa2-079116103057 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:28.998038 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb3ad4-896c-4377-afa2-079116103057 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:28.998316 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb3ad4-896c-4377-afa2-079116103057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:29.016043 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1215/2416] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:39:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:39:29.016043 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4ba28-b354-4cf9-bb3f-fc1d56ba567c None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:29.016043 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4ba28-b354-4cf9-bb3f-fc1d56ba567c None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:29.016043 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4ba28-b354-4cf9-bb3f-fc1d56ba567c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:29.026994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-cddb3ad4-896c-4377-afa2-079116103057 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:29.028335 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-cddb3ad4-896c-4377-afa2-079116103057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:29.036592 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-1ba4ba28-b354-4cf9-bb3f-fc1d56ba567c None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:29.037964 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-1ba4ba28-b354-4cf9-bb3f-fc1d56ba567c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:29.041662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1202/2417] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:39:29.041662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb3ad4-896c-4377-afa2-079116103057 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:29.047915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-cddb3ad4-896c-4377-afa2-079116103057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:29.055908 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1216/2418] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:39:29.057257 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4ba28-b354-4cf9-bb3f-fc1d56ba567c None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:29.061643 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb3ad4-896c-4377-afa2-079116103057 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:29.067891 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4ba28-b354-4cf9-bb3f-fc1d56ba567c None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:29.075462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa86f4e-5604-4001-b96d-114b24dfa9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:29.078804 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa86f4e-5604-4001-b96d-114b24dfa9f5 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:29.078804 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa86f4e-5604-4001-b96d-114b24dfa9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:29.078804 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa86f4e-5604-4001-b96d-114b24dfa9f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:29.078804 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa86f4e-5604-4001-b96d-114b24dfa9f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:29.079619 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa86f4e-5604-4001-b96d-114b24dfa9f5 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:29.081217 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce8ef0-edcc-459d-93d0-9886fcb6aa27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:29.082124 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce8ef0-edcc-459d-93d0-9886fcb6aa27 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:29.082565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce8ef0-edcc-459d-93d0-9886fcb6aa27 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:29.082943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce8ef0-edcc-459d-93d0-9886fcb6aa27 admin admin] PATH_INFO: `/v3/projects/f9ac4c1e5fc641049be06afa40b4c6ea/users/fcd4b6da197d4717a6cd691f53023058/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:29.100834 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ce8ef0-edcc-459d-93d0-9886fcb6aa27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9ac4c1e5fc641049be06afa40b4c6ea, user_id=fcd4b6da197d4717a6cd691f53023058, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:29.104315 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ce8ef0-edcc-459d-93d0-9886fcb6aa27 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:29.191433 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1203/2419] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:39:29 2022] PUT /identity/v3/projects/f9ac4c1e5fc641049be06afa40b4c6ea/users/fcd4b6da197d4717a6cd691f53023058/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:29.192283 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce8ef0-edcc-459d-93d0-9886fcb6aa27 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:29.194055 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a2ce8ef0-edcc-459d-93d0-9886fcb6aa27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:29.217160 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=86,keystone:INSERT=3,keystone:DELETE=11 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:29.217962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1217/2420] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:39:29 2022] POST /identity/v3/projects => generated 355 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:39:29.221267 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa301dd2-9a3f-434f-9bde-7b4a17d3c349 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:29.221267 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa301dd2-9a3f-434f-9bde-7b4a17d3c349 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:29.221267 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa301dd2-9a3f-434f-9bde-7b4a17d3c349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:29.244788 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-aa301dd2-9a3f-434f-9bde-7b4a17d3c349 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:29.246308 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-aa301dd2-9a3f-434f-9bde-7b4a17d3c349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:29.272308 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce8ef0-edcc-459d-93d0-9886fcb6aa27 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:29.324750 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1218/2421] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:39:29.325652 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa301dd2-9a3f-434f-9bde-7b4a17d3c349 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:29.338482 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa301dd2-9a3f-434f-9bde-7b4a17d3c349 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:29.346731 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:29.348326 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dee4a6-f6dd-4521-8402-8075cadcc999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:29.349134 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dee4a6-f6dd-4521-8402-8075cadcc999 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:29.349407 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dee4a6-f6dd-4521-8402-8075cadcc999 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:29.349671 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dee4a6-f6dd-4521-8402-8075cadcc999 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:29.350318 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dee4a6-f6dd-4521-8402-8075cadcc999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:29.352177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dee4a6-f6dd-4521-8402-8075cadcc999 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:29.358754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86335f09-172b-49a4-98c6-b946ef3366c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:29.359379 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86335f09-172b-49a4-98c6-b946ef3366c8 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:29.359654 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86335f09-172b-49a4-98c6-b946ef3366c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:29.359923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86335f09-172b-49a4-98c6-b946ef3366c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:29.365328 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1219/2422] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:39:29.366380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dee4a6-f6dd-4521-8402-8075cadcc999 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:29.377119 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86335f09-172b-49a4-98c6-b946ef3366c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:29.379403 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86335f09-172b-49a4-98c6-b946ef3366c8 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:29.381170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dee4a6-f6dd-4521-8402-8075cadcc999 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:29.386932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a64da-5c1f-40f8-a4ed-3a478d2996f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:29.387586 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a64da-5c1f-40f8-a4ed-3a478d2996f8 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:29.387822 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a64da-5c1f-40f8-a4ed-3a478d2996f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:29.388087 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a64da-5c1f-40f8-a4ed-3a478d2996f8 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:29.388880 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a64da-5c1f-40f8-a4ed-3a478d2996f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:29.391176 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a64da-5c1f-40f8-a4ed-3a478d2996f8 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:29.395486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1204/2423] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:39:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:39:29.396339 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86335f09-172b-49a4-98c6-b946ef3366c8 service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:29.408198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86335f09-172b-49a4-98c6-b946ef3366c8 service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:29.414775 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14143fd2-5179-49e8-a6c0-33f183b636c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:29.416346 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14143fd2-5179-49e8-a6c0-33f183b636c1 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:29.417109 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14143fd2-5179-49e8-a6c0-33f183b636c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:29.417298 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14143fd2-5179-49e8-a6c0-33f183b636c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:29.418234 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14143fd2-5179-49e8-a6c0-33f183b636c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:39:29.418559 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14143fd2-5179-49e8-a6c0-33f183b636c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:29.421042 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14143fd2-5179-49e8-a6c0-33f183b636c1 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:29.440990 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1205/2424] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:39:29.440990 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14143fd2-5179-49e8-a6c0-33f183b636c1 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:29.453686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14143fd2-5179-49e8-a6c0-33f183b636c1 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:29.462006 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48024de-e645-4dc1-82cf-a6b711379fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:29.463378 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48024de-e645-4dc1-82cf-a6b711379fc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:29.463801 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48024de-e645-4dc1-82cf-a6b711379fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:29.464166 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48024de-e645-4dc1-82cf-a6b711379fc2 admin admin] PATH_INFO: `/v3/projects/f9ac4c1e5fc641049be06afa40b4c6ea/users/fcd4b6da197d4717a6cd691f53023058/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:29.466713 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48024de-e645-4dc1-82cf-a6b711379fc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9ac4c1e5fc641049be06afa40b4c6ea, user_id=fcd4b6da197d4717a6cd691f53023058, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:29.469993 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48024de-e645-4dc1-82cf-a6b711379fc2 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:29.492561 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1220/2425] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:39:29 2022] POST /identity/v3/users => generated 278 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:39:29.492561 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a64da-5c1f-40f8-a4ed-3a478d2996f8 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:29.499197 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1206/2426] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:39:29 2022] PUT /identity/v3/projects/f9ac4c1e5fc641049be06afa40b4c6ea/users/fcd4b6da197d4717a6cd691f53023058/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:29.501657 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48024de-e645-4dc1-82cf-a6b711379fc2 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:29.516993 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a64da-5c1f-40f8-a4ed-3a478d2996f8 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:29.600282 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48024de-e645-4dc1-82cf-a6b711379fc2 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:29.604389 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a04c54-5d6d-48d6-a57e-69b60e18837c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:29.605332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a04c54-5d6d-48d6-a57e-69b60e18837c admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:29.606215 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a04c54-5d6d-48d6-a57e-69b60e18837c admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:29.606917 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a04c54-5d6d-48d6-a57e-69b60e18837c admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:29.607376 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a595e73-c3bd-4e6e-ad35-9d47d5c29a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:29.607914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a04c54-5d6d-48d6-a57e-69b60e18837c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:29.608298 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a595e73-c3bd-4e6e-ad35-9d47d5c29a3d admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:29.608900 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a595e73-c3bd-4e6e-ad35-9d47d5c29a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:29.609336 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a595e73-c3bd-4e6e-ad35-9d47d5c29a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:29.610018 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a595e73-c3bd-4e6e-ad35-9d47d5c29a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:29.610694 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a04c54-5d6d-48d6-a57e-69b60e18837c admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:29.611688 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a595e73-c3bd-4e6e-ad35-9d47d5c29a3d admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:29.623434 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1207/2427] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:39:29.624771 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9deed0a-407d-470c-9345-e52e38bdd4c1 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:29.625122 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9deed0a-407d-470c-9345-e52e38bdd4c1 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:29.625394 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9deed0a-407d-470c-9345-e52e38bdd4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:29.654292 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-e9deed0a-407d-470c-9345-e52e38bdd4c1 None None] MFA Rules not processed for user `fcd4b6da197d4717a6cd691f53023058`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:29.655588 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-e9deed0a-407d-470c-9345-e52e38bdd4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:29.725646 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1221/2428] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:39:29 2022] POST /identity/v3/projects => generated 397 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:39:29.726824 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a04c54-5d6d-48d6-a57e-69b60e18837c admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:29.753713 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1208/2429] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:39:29 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:39:29.755256 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9deed0a-407d-470c-9345-e52e38bdd4c1 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:29.780728 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9deed0a-407d-470c-9345-e52e38bdd4c1 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:29.814887 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:29.850932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a04c54-5d6d-48d6-a57e-69b60e18837c admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:29.857511 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ecdebc-6016-4ec0-869a-c8d478813bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:29.858501 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ecdebc-6016-4ec0-869a-c8d478813bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:29.858886 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ecdebc-6016-4ec0-869a-c8d478813bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:29.859262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ecdebc-6016-4ec0-869a-c8d478813bb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:29.860023 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ecdebc-6016-4ec0-869a-c8d478813bb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:29.864002 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ecdebc-6016-4ec0-869a-c8d478813bb9 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:29.877921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1222/2430] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:39:29.878919 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ecdebc-6016-4ec0-869a-c8d478813bb9 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:29.881102 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-85ecdebc-6016-4ec0-869a-c8d478813bb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:29.909616 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3b664-c7fa-4c75-8e0c-f170867edd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:29.910273 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3b664-c7fa-4c75-8e0c-f170867edd20 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:29.910596 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3b664-c7fa-4c75-8e0c-f170867edd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:29.910876 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3b664-c7fa-4c75-8e0c-f170867edd20 admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:29.911694 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced3b664-c7fa-4c75-8e0c-f170867edd20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:29.916898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced3b664-c7fa-4c75-8e0c-f170867edd20 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:29.985457 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:29.997202 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ecdebc-6016-4ec0-869a-c8d478813bb9 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:30.001824 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb56f0-d6fd-4fb5-98c1-2928474ec9dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:30.002481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb56f0-d6fd-4fb5-98c1-2928474ec9dd service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:30.002792 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb56f0-d6fd-4fb5-98c1-2928474ec9dd service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:30.003039 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb56f0-d6fd-4fb5-98c1-2928474ec9dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:30.050467 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1209/2431] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:39:29 2022] POST /identity/v3/users => generated 299 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:39:30.053252 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3b664-c7fa-4c75-8e0c-f170867edd20 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:30.064429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3b664-c7fa-4c75-8e0c-f170867edd20 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:30.069561 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74186653-c190-4803-bd15-c68ad56e1c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:30.070541 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74186653-c190-4803-bd15-c68ad56e1c05 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:30.070826 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74186653-c190-4803-bd15-c68ad56e1c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:30.071102 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74186653-c190-4803-bd15-c68ad56e1c05 admin admin] PATH_INFO: `/v3/projects/25ca49c56fdc409b993ec34f5d794044/users/82856c346db54659aa29107daadceafe/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:30.079292 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb56f0-d6fd-4fb5-98c1-2928474ec9dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:30.080787 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb56f0-d6fd-4fb5-98c1-2928474ec9dd service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:30.092520 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1223/2432] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:39:29 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:39:30.097872 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74186653-c190-4803-bd15-c68ad56e1c05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25ca49c56fdc409b993ec34f5d794044, user_id=82856c346db54659aa29107daadceafe, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:30.099658 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb56f0-d6fd-4fb5-98c1-2928474ec9dd service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:30.100859 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74186653-c190-4803-bd15-c68ad56e1c05 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:30.120798 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb56f0-d6fd-4fb5-98c1-2928474ec9dd service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:30.130685 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb62278-aa17-4c0d-90e2-841ceacb014a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:30.138918 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb62278-aa17-4c0d-90e2-841ceacb014a admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:30.139363 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb62278-aa17-4c0d-90e2-841ceacb014a admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:30.139638 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb62278-aa17-4c0d-90e2-841ceacb014a admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:30.140517 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb62278-aa17-4c0d-90e2-841ceacb014a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:30.143618 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb62278-aa17-4c0d-90e2-841ceacb014a admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:30.155105 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1224/2433] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:39:30.156200 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb62278-aa17-4c0d-90e2-841ceacb014a admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:30.159262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-5eb62278-aa17-4c0d-90e2-841ceacb014a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:30.172645 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb62278-aa17-4c0d-90e2-841ceacb014a admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:30.180101 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1210/2434] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:39:30 2022] PUT /identity/v3/projects/25ca49c56fdc409b993ec34f5d794044/users/82856c346db54659aa29107daadceafe/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:30.180820 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74186653-c190-4803-bd15-c68ad56e1c05 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:30.232423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6511dd9-8b62-40d6-b9ed-70069caf83c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:30.233052 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6511dd9-8b62-40d6-b9ed-70069caf83c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:30.233322 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6511dd9-8b62-40d6-b9ed-70069caf83c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:30.233595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6511dd9-8b62-40d6-b9ed-70069caf83c1 admin admin] PATH_INFO: `/v3/projects/c5578bf9341646b9b4665cc0c1b13250` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:30.234815 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6511dd9-8b62-40d6-b9ed-70069caf83c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5578bf9341646b9b4665cc0c1b13250)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:30.236729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6511dd9-8b62-40d6-b9ed-70069caf83c1 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:30.261362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74186653-c190-4803-bd15-c68ad56e1c05 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:30.268473 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc3d6d3-346f-4b78-90d9-6af0cdbe4ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:30.269233 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3d6d3-346f-4b78-90d9-6af0cdbe4ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:30.270273 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3d6d3-346f-4b78-90d9-6af0cdbe4ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:30.270273 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3d6d3-346f-4b78-90d9-6af0cdbe4ca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:30.271146 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc3d6d3-346f-4b78-90d9-6af0cdbe4ca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:30.273851 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc3d6d3-346f-4b78-90d9-6af0cdbe4ca7 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:30.297537 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1211/2435] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:39:30.297537 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5cc14f-e8fb-4f7c-8c9c-c9fd28d5d45a None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:30.297537 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5cc14f-e8fb-4f7c-8c9c-c9fd28d5d45a None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:30.297537 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5cc14f-e8fb-4f7c-8c9c-c9fd28d5d45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:30.340874 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-fa5cc14f-e8fb-4f7c-8c9c-c9fd28d5d45a None None] MFA Rules not processed for user `82856c346db54659aa29107daadceafe`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:30.342422 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-fa5cc14f-e8fb-4f7c-8c9c-c9fd28d5d45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:30.375353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:30.457673 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a6511dd9-8b62-40d6-b9ed-70069caf83c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5578bf9341646b9b4665cc0c1b13250', 'request_id': 'req-a6511dd9-8b62-40d6-b9ed-70069caf83c1'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:30.477578 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1212/2436] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:39:30 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:39:30.484842 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5cc14f-e8fb-4f7c-8c9c-c9fd28d5d45a None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:30.494931 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5cc14f-e8fb-4f7c-8c9c-c9fd28d5d45a None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:30.554533 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa1bd4-24fb-46b9-b3f1-6ce3407246f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:30.555116 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa1bd4-24fb-46b9-b3f1-6ce3407246f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:30.555387 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa1bd4-24fb-46b9-b3f1-6ce3407246f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:30.555657 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa1bd4-24fb-46b9-b3f1-6ce3407246f5 admin admin] PATH_INFO: `/v3/projects/87226efbc8db4654b947e386737a1ccd/users/3156139a06844f329f4820d054966543/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:30.570051 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa1bd4-24fb-46b9-b3f1-6ce3407246f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87226efbc8db4654b947e386737a1ccd, user_id=3156139a06844f329f4820d054966543, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:30.572805 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa1bd4-24fb-46b9-b3f1-6ce3407246f5 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:30.577345 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a6511dd9-8b62-40d6-b9ed-70069caf83c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5578bf9341646b9b4665cc0c1b13250', 'request_id': 'req-a6511dd9-8b62-40d6-b9ed-70069caf83c1'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:30.584717 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a6511dd9-8b62-40d6-b9ed-70069caf83c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5578bf9341646b9b4665cc0c1b13250', 'request_id': 'req-a6511dd9-8b62-40d6-b9ed-70069caf83c1'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:30.598828 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a6511dd9-8b62-40d6-b9ed-70069caf83c1 admin admin] The token cache is being invalidate because project c5578bf9341646b9b4665cc0c1b13250 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:39:30.599207 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a6511dd9-8b62-40d6-b9ed-70069caf83c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:30.600035 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a6511dd9-8b62-40d6-b9ed-70069caf83c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:30.601572 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1225/2437] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:39:30 2022] DELETE /identity/v3/projects/c5578bf9341646b9b4665cc0c1b13250 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:30.605451 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6511dd9-8b62-40d6-b9ed-70069caf83c1 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:30.605451 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a6511dd9-8b62-40d6-b9ed-70069caf83c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:30.623573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:30.663368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6511dd9-8b62-40d6-b9ed-70069caf83c1 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:30.668613 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb760cb-bc3f-4f0b-93b2-61dba78743f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:30.669251 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb760cb-bc3f-4f0b-93b2-61dba78743f4 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:30.669529 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb760cb-bc3f-4f0b-93b2-61dba78743f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:30.669804 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb760cb-bc3f-4f0b-93b2-61dba78743f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:30.671642 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-bfb760cb-bc3f-4f0b-93b2-61dba78743f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:30.672765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1213/2438] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:39:30 2022] PUT /identity/v3/projects/87226efbc8db4654b947e386737a1ccd/users/3156139a06844f329f4820d054966543/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:30.684694 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa1bd4-24fb-46b9-b3f1-6ce3407246f5 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:30.686743 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-b0aa1bd4-24fb-46b9-b3f1-6ce3407246f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:30.743907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb760cb-bc3f-4f0b-93b2-61dba78743f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:30.745762 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb760cb-bc3f-4f0b-93b2-61dba78743f4 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:30.756082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1226/2439] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:39:30 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:39:30.762229 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb760cb-bc3f-4f0b-93b2-61dba78743f4 service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:30.763711 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-bfb760cb-bc3f-4f0b-93b2-61dba78743f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:30.778313 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb760cb-bc3f-4f0b-93b2-61dba78743f4 service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:30.780392 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa1bd4-24fb-46b9-b3f1-6ce3407246f5 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:30.783297 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bfcc1-b3ca-49b9-aa14-f911433ab834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:30.784079 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bfcc1-b3ca-49b9-aa14-f911433ab834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:30.784362 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bfcc1-b3ca-49b9-aa14-f911433ab834 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:30.784683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bfcc1-b3ca-49b9-aa14-f911433ab834 admin admin] PATH_INFO: `/v3/users/95ae0411267f47718e44a2c0cc4b2d9d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:30.786072 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5bfcc1-b3ca-49b9-aa14-f911433ab834 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95ae0411267f47718e44a2c0cc4b2d9d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:30.788660 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5bfcc1-b3ca-49b9-aa14-f911433ab834 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:30.804877 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3974bf7-d50b-45da-8395-d7502892c784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:30.805850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3974bf7-d50b-45da-8395-d7502892c784 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:30.807266 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3974bf7-d50b-45da-8395-d7502892c784 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:30.807570 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3974bf7-d50b-45da-8395-d7502892c784 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:30.808245 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3974bf7-d50b-45da-8395-d7502892c784 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:30.811072 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3974bf7-d50b-45da-8395-d7502892c784 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:30.963908 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1214/2440] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:39:30 2022] POST /identity/v3/projects => generated 355 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:39:30.963908 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781ddf2-4159-4081-bfb8-63490a4847fc None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:30.963908 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781ddf2-4159-4081-bfb8-63490a4847fc None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:30.963908 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781ddf2-4159-4081-bfb8-63490a4847fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:31.006711 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-4781ddf2-4159-4081-bfb8-63490a4847fc None None] MFA Rules not processed for user `3156139a06844f329f4820d054966543`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:31.007966 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-4781ddf2-4159-4081-bfb8-63490a4847fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:31.035227 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cc5bfcc1-b3ca-49b9-aa14-f911433ab834 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95ae0411267f47718e44a2c0cc4b2d9d'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:31.050201 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:31.068769 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cc5bfcc1-b3ca-49b9-aa14-f911433ab834 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95ae0411267f47718e44a2c0cc4b2d9d'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:31.080782 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cc5bfcc1-b3ca-49b9-aa14-f911433ab834 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95ae0411267f47718e44a2c0cc4b2d9d'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:31.144232 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1215/2441] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:39:30 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:39:31.145317 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781ddf2-4159-4081-bfb8-63490a4847fc None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:31.176632 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cc5bfcc1-b3ca-49b9-aa14-f911433ab834 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95ae0411267f47718e44a2c0cc4b2d9d'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:31.176632 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cc5bfcc1-b3ca-49b9-aa14-f911433ab834 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95ae0411267f47718e44a2c0cc4b2d9d'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:31.181978 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1227/2442] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:39:30 2022] DELETE /identity/v3/users/95ae0411267f47718e44a2c0cc4b2d9d => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:31.182342 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bfcc1-b3ca-49b9-aa14-f911433ab834 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:31.184700 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-cc5bfcc1-b3ca-49b9-aa14-f911433ab834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:31.270904 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:31.295196 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781ddf2-4159-4081-bfb8-63490a4847fc None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:31.339633 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bfcc1-b3ca-49b9-aa14-f911433ab834 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:31.346344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fda8b3-3322-4dd2-85f9-b870f7299dad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:31.347682 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fda8b3-3322-4dd2-85f9-b870f7299dad service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:31.348063 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fda8b3-3322-4dd2-85f9-b870f7299dad service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:31.348425 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fda8b3-3322-4dd2-85f9-b870f7299dad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:31.351273 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-51fda8b3-3322-4dd2-85f9-b870f7299dad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:31.442223 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae985c-ccd6-4114-9dd1-01d6ecae1051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:31.442906 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae985c-ccd6-4114-9dd1-01d6ecae1051 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:31.443182 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae985c-ccd6-4114-9dd1-01d6ecae1051 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:31.443455 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae985c-ccd6-4114-9dd1-01d6ecae1051 admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:31.444353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae985c-ccd6-4114-9dd1-01d6ecae1051 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:31.451701 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae985c-ccd6-4114-9dd1-01d6ecae1051 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:31.485099 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fda8b3-3322-4dd2-85f9-b870f7299dad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:31.487082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fda8b3-3322-4dd2-85f9-b870f7299dad service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:31.506382 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1228/2443] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:39:31 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:39:31.595082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1216/2444] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:39:31 2022] POST /identity/v3/users => generated 284 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:39:31.596579 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fda8b3-3322-4dd2-85f9-b870f7299dad service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:31.598133 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-51fda8b3-3322-4dd2-85f9-b870f7299dad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:31.612863 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fda8b3-3322-4dd2-85f9-b870f7299dad service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:31.619746 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243fdfc5-9272-427b-a28f-ad2dc53591e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:31.620523 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fdfc5-9272-427b-a28f-ad2dc53591e6 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:31.620863 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fdfc5-9272-427b-a28f-ad2dc53591e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:31.621831 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fdfc5-9272-427b-a28f-ad2dc53591e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:31.622582 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243fdfc5-9272-427b-a28f-ad2dc53591e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:31.628026 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243fdfc5-9272-427b-a28f-ad2dc53591e6 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:31.642898 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1229/2445] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:39:31.643905 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae985c-ccd6-4114-9dd1-01d6ecae1051 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:31.660547 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae985c-ccd6-4114-9dd1-01d6ecae1051 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:31.673195 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621f9f8-19ba-42b0-8be9-ea7f83a8d800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:31.673878 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621f9f8-19ba-42b0-8be9-ea7f83a8d800 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:31.674224 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621f9f8-19ba-42b0-8be9-ea7f83a8d800 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:31.674573 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621f9f8-19ba-42b0-8be9-ea7f83a8d800 admin admin] PATH_INFO: `/v3/projects/5ceaea556ace4ac4aec6e9fe964b2e12/users/8aba47a3a35b41419560e25cf3e4ae42/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:31.702035 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621f9f8-19ba-42b0-8be9-ea7f83a8d800 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ceaea556ace4ac4aec6e9fe964b2e12, user_id=8aba47a3a35b41419560e25cf3e4ae42, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:31.706295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621f9f8-19ba-42b0-8be9-ea7f83a8d800 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:31.779805 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1217/2446] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:39:31 2022] PUT /identity/v3/projects/5ceaea556ace4ac4aec6e9fe964b2e12/users/8aba47a3a35b41419560e25cf3e4ae42/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:39:31.783682 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243fdfc5-9272-427b-a28f-ad2dc53591e6 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:31.794493 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243fdfc5-9272-427b-a28f-ad2dc53591e6 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:31.864838 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03673229-0873-4b99-b927-645776f2996c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:31.865444 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03673229-0873-4b99-b927-645776f2996c admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:31.865733 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03673229-0873-4b99-b927-645776f2996c admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:31.866004 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03673229-0873-4b99-b927-645776f2996c admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:31.866777 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03673229-0873-4b99-b927-645776f2996c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:31.868877 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03673229-0873-4b99-b927-645776f2996c admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:31.881208 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1230/2447] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:39:31.886119 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621f9f8-19ba-42b0-8be9-ea7f83a8d800 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:31.902907 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621f9f8-19ba-42b0-8be9-ea7f83a8d800 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:31.903969 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03673229-0873-4b99-b927-645776f2996c admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:31.905741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-03673229-0873-4b99-b927-645776f2996c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:31.910470 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da011c-b922-4651-a0d1-12cd73db75bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:31.911561 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da011c-b922-4651-a0d1-12cd73db75bf admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:31.911844 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da011c-b922-4651-a0d1-12cd73db75bf admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:31.912119 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da011c-b922-4651-a0d1-12cd73db75bf admin admin] PATH_INFO: `/v3/projects/5ceaea556ace4ac4aec6e9fe964b2e12/users/8aba47a3a35b41419560e25cf3e4ae42/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:31.914928 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da011c-b922-4651-a0d1-12cd73db75bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ceaea556ace4ac4aec6e9fe964b2e12, user_id=8aba47a3a35b41419560e25cf3e4ae42, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:31.917695 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da011c-b922-4651-a0d1-12cd73db75bf admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:31.919585 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03673229-0873-4b99-b927-645776f2996c admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:31.924445 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ddb53-06e7-42a9-81e6-bd055d9a4c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:31.925068 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ddb53-06e7-42a9-81e6-bd055d9a4c4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:31.925383 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ddb53-06e7-42a9-81e6-bd055d9a4c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:31.925598 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ddb53-06e7-42a9-81e6-bd055d9a4c4c admin admin] PATH_INFO: `/v3/projects/3ff5c984f6264d2fb023e01e80b520d4` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:31.926803 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ddb53-06e7-42a9-81e6-bd055d9a4c4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ff5c984f6264d2fb023e01e80b520d4)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:31.928702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ddb53-06e7-42a9-81e6-bd055d9a4c4c admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:31.947279 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1218/2448] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:39:31 2022] PUT /identity/v3/projects/5ceaea556ace4ac4aec6e9fe964b2e12/users/8aba47a3a35b41419560e25cf3e4ae42/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:31.961024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c2c73-1e3c-4720-9bfd-284c34b7240f None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:31.961348 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c2c73-1e3c-4720-9bfd-284c34b7240f None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:31.961754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c2c73-1e3c-4720-9bfd-284c34b7240f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:31.997076 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-db9c2c73-1e3c-4720-9bfd-284c34b7240f None None] MFA Rules not processed for user `8aba47a3a35b41419560e25cf3e4ae42`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:31.998434 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-db9c2c73-1e3c-4720-9bfd-284c34b7240f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:32.008509 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:32.086456 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-097ddb53-06e7-42a9-81e6-bd055d9a4c4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ff5c984f6264d2fb023e01e80b520d4', 'request_id': 'req-097ddb53-06e7-42a9-81e6-bd055d9a4c4c'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:32.090537 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1219/2449] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:39:31 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:39:32.099322 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c2c73-1e3c-4720-9bfd-284c34b7240f None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:32.159066 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c2c73-1e3c-4720-9bfd-284c34b7240f None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:32.198924 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-097ddb53-06e7-42a9-81e6-bd055d9a4c4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ff5c984f6264d2fb023e01e80b520d4', 'request_id': 'req-097ddb53-06e7-42a9-81e6-bd055d9a4c4c'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:32.215437 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-097ddb53-06e7-42a9-81e6-bd055d9a4c4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ff5c984f6264d2fb023e01e80b520d4', 'request_id': 'req-097ddb53-06e7-42a9-81e6-bd055d9a4c4c'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:32.219484 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f720a7-6339-4549-b851-2b0b5696c875 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:32.220109 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f720a7-6339-4549-b851-2b0b5696c875 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:32.220387 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f720a7-6339-4549-b851-2b0b5696c875 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:32.220658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f720a7-6339-4549-b851-2b0b5696c875 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:32.234365 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-097ddb53-06e7-42a9-81e6-bd055d9a4c4c admin admin] The token cache is being invalidate because project 3ff5c984f6264d2fb023e01e80b520d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:39:32.241796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-097ddb53-06e7-42a9-81e6-bd055d9a4c4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:32.241796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-097ddb53-06e7-42a9-81e6-bd055d9a4c4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:32.241796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1231/2450] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:39:31 2022] DELETE /identity/v3/projects/3ff5c984f6264d2fb023e01e80b520d4 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:39:32.244116 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7c35e-c4ef-43e9-b12b-dae4b2124d08 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:32.244116 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7c35e-c4ef-43e9-b12b-dae4b2124d08 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:32.244403 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7c35e-c4ef-43e9-b12b-dae4b2124d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:32.253657 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:32.267981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-3af7c35e-c4ef-43e9-b12b-dae4b2124d08 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:32.269470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-3af7c35e-c4ef-43e9-b12b-dae4b2124d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:32.294946 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f720a7-6339-4549-b851-2b0b5696c875 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:32.296438 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f720a7-6339-4549-b851-2b0b5696c875 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:32.308602 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1220/2451] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:39:32 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:39:32.350291 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1232/2452] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:39:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:39:32.358318 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793002cf-7fe6-4619-b715-eae100a65bc8 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:32.358635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793002cf-7fe6-4619-b715-eae100a65bc8 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:32.358938 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793002cf-7fe6-4619-b715-eae100a65bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:32.399560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-793002cf-7fe6-4619-b715-eae100a65bc8 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:32.401516 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-793002cf-7fe6-4619-b715-eae100a65bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:32.442096 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1221/2453] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:39:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:39:32.446419 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7c35e-c4ef-43e9-b12b-dae4b2124d08 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:32.470222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7c35e-c4ef-43e9-b12b-dae4b2124d08 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:32.475748 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f34cf9-dc59-4189-848e-c04acd32d09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:32.476449 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f34cf9-dc59-4189-848e-c04acd32d09a admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:32.476999 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f34cf9-dc59-4189-848e-c04acd32d09a admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:32.477383 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f34cf9-dc59-4189-848e-c04acd32d09a admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:32.478711 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f34cf9-dc59-4189-848e-c04acd32d09a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:39:32.479571 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f34cf9-dc59-4189-848e-c04acd32d09a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:32.481952 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f34cf9-dc59-4189-848e-c04acd32d09a admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:32.517121 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1233/2454] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:39:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:39:32.521244 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793002cf-7fe6-4619-b715-eae100a65bc8 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:32.531284 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793002cf-7fe6-4619-b715-eae100a65bc8 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:32.538550 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03278b77-2620-4433-84b4-a73e03b47c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:32.538550 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03278b77-2620-4433-84b4-a73e03b47c78 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:32.538550 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03278b77-2620-4433-84b4-a73e03b47c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:32.538550 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03278b77-2620-4433-84b4-a73e03b47c78 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:32.539053 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03278b77-2620-4433-84b4-a73e03b47c78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:32.541262 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03278b77-2620-4433-84b4-a73e03b47c78 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:32.615701 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f34cf9-dc59-4189-848e-c04acd32d09a admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:32.623499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-04f34cf9-dc59-4189-848e-c04acd32d09a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:32.641497 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1222/2455] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:39:32 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:39:32.645629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03278b77-2620-4433-84b4-a73e03b47c78 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:32.646060 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f34cf9-dc59-4189-848e-c04acd32d09a admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:32.727016 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad42013-3feb-435d-b96b-81a0f4314335 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:32.728109 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad42013-3feb-435d-b96b-81a0f4314335 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:32.728750 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad42013-3feb-435d-b96b-81a0f4314335 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:32.728859 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad42013-3feb-435d-b96b-81a0f4314335 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:32.731927 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-1ad42013-3feb-435d-b96b-81a0f4314335 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:32.746191 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03278b77-2620-4433-84b4-a73e03b47c78 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:32.751445 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f413311-ef0c-42f0-8d5e-3961eb226506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:32.752368 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f413311-ef0c-42f0-8d5e-3961eb226506 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:32.752789 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f413311-ef0c-42f0-8d5e-3961eb226506 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:32.753204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f413311-ef0c-42f0-8d5e-3961eb226506 admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:32.753983 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f413311-ef0c-42f0-8d5e-3961eb226506 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:32.756290 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f413311-ef0c-42f0-8d5e-3961eb226506 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:32.805713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad42013-3feb-435d-b96b-81a0f4314335 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:32.807841 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad42013-3feb-435d-b96b-81a0f4314335 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:32.817277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1234/2456] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:39:32 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:39:32.874697 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:32.878955 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1223/2457] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:39:32 2022] POST /identity/v3/users => generated 293 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:39:32.883317 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad42013-3feb-435d-b96b-81a0f4314335 service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:32.894765 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad42013-3feb-435d-b96b-81a0f4314335 service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:32.900193 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d099536-cb6c-4c92-89bf-b69a4c20fffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:32.900959 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d099536-cb6c-4c92-89bf-b69a4c20fffe admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:32.901345 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d099536-cb6c-4c92-89bf-b69a4c20fffe admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:32.901620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d099536-cb6c-4c92-89bf-b69a4c20fffe admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:32.902282 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d099536-cb6c-4c92-89bf-b69a4c20fffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:32.904082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d099536-cb6c-4c92-89bf-b69a4c20fffe admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:32.914380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1235/2458] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:39:32.919429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f413311-ef0c-42f0-8d5e-3961eb226506 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:32.929490 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f413311-ef0c-42f0-8d5e-3961eb226506 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:32.934380 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2bcd4-e925-47a7-803b-280dc4873265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:32.935000 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2bcd4-e925-47a7-803b-280dc4873265 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:32.935276 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2bcd4-e925-47a7-803b-280dc4873265 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:32.935545 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2bcd4-e925-47a7-803b-280dc4873265 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:32.936233 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f2bcd4-e925-47a7-803b-280dc4873265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:32.937817 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f2bcd4-e925-47a7-803b-280dc4873265 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:32.948129 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1224/2459] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:39:32.952475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d099536-cb6c-4c92-89bf-b69a4c20fffe admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:32.963953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d099536-cb6c-4c92-89bf-b69a4c20fffe admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:32.968882 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b00e7ff-548f-47b0-9693-193f244547fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:32.969563 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00e7ff-548f-47b0-9693-193f244547fc admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:32.969837 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00e7ff-548f-47b0-9693-193f244547fc admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:32.970103 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00e7ff-548f-47b0-9693-193f244547fc admin admin] PATH_INFO: `/v3/projects/a50199051ad04d44aaa79918fa839beb/users/f177a8a49f274929a40ffeb7a551c003/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:32.985873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b00e7ff-548f-47b0-9693-193f244547fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a50199051ad04d44aaa79918fa839beb, user_id=f177a8a49f274929a40ffeb7a551c003, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:32.988592 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b00e7ff-548f-47b0-9693-193f244547fc admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:33.084505 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1236/2460] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:39:32 2022] PUT /identity/v3/projects/a50199051ad04d44aaa79918fa839beb/users/f177a8a49f274929a40ffeb7a551c003/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:33.095351 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b125dc4-5aff-4003-90bd-c24d837e3c21 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:33.095920 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b125dc4-5aff-4003-90bd-c24d837e3c21 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:33.096296 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b125dc4-5aff-4003-90bd-c24d837e3c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:33.134744 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-3b125dc4-5aff-4003-90bd-c24d837e3c21 None None] MFA Rules not processed for user `f177a8a49f274929a40ffeb7a551c003`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:33.135974 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-3b125dc4-5aff-4003-90bd-c24d837e3c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:33.224330 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1225/2461] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:39:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:39:33.229372 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b00e7ff-548f-47b0-9693-193f244547fc admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:33.294303 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b00e7ff-548f-47b0-9693-193f244547fc admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:33.299055 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8163a6-57d1-4c62-87d6-d6d28bfe656e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:33.299715 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8163a6-57d1-4c62-87d6-d6d28bfe656e admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:33.300057 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8163a6-57d1-4c62-87d6-d6d28bfe656e admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:33.300338 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8163a6-57d1-4c62-87d6-d6d28bfe656e admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:33.301008 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8163a6-57d1-4c62-87d6-d6d28bfe656e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:33.302937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8163a6-57d1-4c62-87d6-d6d28bfe656e admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:33.430243 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1237/2462] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:39:33 2022] POST /identity/v3/projects => generated 383 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:39:33.432036 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b125dc4-5aff-4003-90bd-c24d837e3c21 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:33.445094 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b125dc4-5aff-4003-90bd-c24d837e3c21 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:33.486338 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:33.525074 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fc2b8-8663-459d-af94-7bba3a9632ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:33.525932 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fc2b8-8663-459d-af94-7bba3a9632ba admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:33.526216 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fc2b8-8663-459d-af94-7bba3a9632ba admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:33.526536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fc2b8-8663-459d-af94-7bba3a9632ba admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:33.527256 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0fc2b8-8663-459d-af94-7bba3a9632ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:33.529194 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0fc2b8-8663-459d-af94-7bba3a9632ba admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:33.556778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:33.650677 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1226/2463] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:39:33 2022] POST /identity/v3/users => generated 292 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:39:33.654730 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8163a6-57d1-4c62-87d6-d6d28bfe656e admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:33.670221 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8163a6-57d1-4c62-87d6-d6d28bfe656e admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:33.674993 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f9f1b-622c-4a2b-b502-68a731246ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:33.675740 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f9f1b-622c-4a2b-b502-68a731246ca9 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:33.676020 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f9f1b-622c-4a2b-b502-68a731246ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:33.676290 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f9f1b-622c-4a2b-b502-68a731246ca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:33.676939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25f9f1b-622c-4a2b-b502-68a731246ca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:33.678558 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25f9f1b-622c-4a2b-b502-68a731246ca9 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:33.690027 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1238/2464] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:39:33.694075 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fc2b8-8663-459d-af94-7bba3a9632ba admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:33.704911 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fc2b8-8663-459d-af94-7bba3a9632ba admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:33.710188 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76055ea-a886-42d0-9670-c55f553eed1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:33.710834 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76055ea-a886-42d0-9670-c55f553eed1b admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:33.711112 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76055ea-a886-42d0-9670-c55f553eed1b admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:33.711382 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76055ea-a886-42d0-9670-c55f553eed1b admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:33.712595 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76055ea-a886-42d0-9670-c55f553eed1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:33.714195 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76055ea-a886-42d0-9670-c55f553eed1b admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:33.725278 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1227/2465] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:39:33.730432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f9f1b-622c-4a2b-b502-68a731246ca9 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:33.742548 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f9f1b-622c-4a2b-b502-68a731246ca9 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:33.749925 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259207e7-902e-42d1-847b-58bea06d6520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:33.750758 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259207e7-902e-42d1-847b-58bea06d6520 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:33.751103 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259207e7-902e-42d1-847b-58bea06d6520 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:33.751443 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259207e7-902e-42d1-847b-58bea06d6520 admin admin] PATH_INFO: `/v3/projects/3597726dc4a7402ba189f4ba26aeaed3/users/fac3cbc4319f4da2bc0baa93fc9c04a7/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:33.770235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259207e7-902e-42d1-847b-58bea06d6520 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3597726dc4a7402ba189f4ba26aeaed3, user_id=fac3cbc4319f4da2bc0baa93fc9c04a7, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:33.773207 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259207e7-902e-42d1-847b-58bea06d6520 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:33.864165 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1239/2466] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:39:33 2022] PUT /identity/v3/projects/3597726dc4a7402ba189f4ba26aeaed3/users/fac3cbc4319f4da2bc0baa93fc9c04a7/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:33.876093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e53cda-df74-472f-8450-73dc9451762f None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:33.876382 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e53cda-df74-472f-8450-73dc9451762f None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:33.876659 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e53cda-df74-472f-8450-73dc9451762f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:33.910780 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-f9e53cda-df74-472f-8450-73dc9451762f None None] MFA Rules not processed for user `fac3cbc4319f4da2bc0baa93fc9c04a7`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:33.911930 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-f9e53cda-df74-472f-8450-73dc9451762f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:33.992399 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1228/2467] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:39:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:39:33.997473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259207e7-902e-42d1-847b-58bea06d6520 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:34.070939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259207e7-902e-42d1-847b-58bea06d6520 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:34.078716 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3aede-79fe-4e7e-b9f4-d877d0665180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:34.079850 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3aede-79fe-4e7e-b9f4-d877d0665180 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:34.080449 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3aede-79fe-4e7e-b9f4-d877d0665180 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:34.080964 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3aede-79fe-4e7e-b9f4-d877d0665180 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:34.081830 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3aede-79fe-4e7e-b9f4-d877d0665180 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:34.084680 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3aede-79fe-4e7e-b9f4-d877d0665180 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:34.182473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1240/2468] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:39:33 2022] POST /identity/v3/projects => generated 383 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:39:34.186962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e53cda-df74-472f-8450-73dc9451762f None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:34.198384 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e53cda-df74-472f-8450-73dc9451762f None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:34.273413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21138b4-5b91-49cd-8a9e-06ce581cfe76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:34.274084 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21138b4-5b91-49cd-8a9e-06ce581cfe76 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:34.274363 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21138b4-5b91-49cd-8a9e-06ce581cfe76 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:34.275342 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21138b4-5b91-49cd-8a9e-06ce581cfe76 admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:34.276103 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21138b4-5b91-49cd-8a9e-06ce581cfe76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:34.278070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21138b4-5b91-49cd-8a9e-06ce581cfe76 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:34.390485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1229/2469] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:39:34 2022] POST /identity/v3/users => generated 298 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:39:34.394647 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3aede-79fe-4e7e-b9f4-d877d0665180 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:34.409368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3aede-79fe-4e7e-b9f4-d877d0665180 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:34.414174 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc0e8a-6a48-4ea9-a0e4-5ceb200b0777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:34.414838 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc0e8a-6a48-4ea9-a0e4-5ceb200b0777 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:34.415115 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc0e8a-6a48-4ea9-a0e4-5ceb200b0777 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:34.416354 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc0e8a-6a48-4ea9-a0e4-5ceb200b0777 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:34.417016 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc0e8a-6a48-4ea9-a0e4-5ceb200b0777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:34.424069 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc0e8a-6a48-4ea9-a0e4-5ceb200b0777 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:34.432175 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1241/2470] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:39:34.436457 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21138b4-5b91-49cd-8a9e-06ce581cfe76 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:34.450751 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21138b4-5b91-49cd-8a9e-06ce581cfe76 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:34.457385 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da87e51-63f5-4d2c-8592-2433f182856c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:34.458067 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da87e51-63f5-4d2c-8592-2433f182856c admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:34.461617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da87e51-63f5-4d2c-8592-2433f182856c admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:34.461617 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da87e51-63f5-4d2c-8592-2433f182856c admin admin] PATH_INFO: `/v3/projects/750d9c8d7db140deab48a6580625ffa7/users/2843cd5feab8472bbbc9ad0930be7176/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:34.479537 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da87e51-63f5-4d2c-8592-2433f182856c admin admin] RBAC: Authorizing `identity:create_grant(project_id=750d9c8d7db140deab48a6580625ffa7, user_id=2843cd5feab8472bbbc9ad0930be7176, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:34.483613 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da87e51-63f5-4d2c-8592-2433f182856c admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:34.614917 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1230/2471] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:39:34 2022] PUT /identity/v3/projects/750d9c8d7db140deab48a6580625ffa7/users/2843cd5feab8472bbbc9ad0930be7176/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:34.618782 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc0e8a-6a48-4ea9-a0e4-5ceb200b0777 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:34.642209 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc0e8a-6a48-4ea9-a0e4-5ceb200b0777 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:34.669937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:34.742250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cc2aa-2c20-485f-9804-4417c2f3a621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:34.743118 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cc2aa-2c20-485f-9804-4417c2f3a621 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:34.743767 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cc2aa-2c20-485f-9804-4417c2f3a621 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:34.744348 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cc2aa-2c20-485f-9804-4417c2f3a621 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:34.745151 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286cc2aa-2c20-485f-9804-4417c2f3a621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:34.747074 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286cc2aa-2c20-485f-9804-4417c2f3a621 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:34.758686 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1242/2472] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:39:34.763672 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da87e51-63f5-4d2c-8592-2433f182856c admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:34.778672 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da87e51-63f5-4d2c-8592-2433f182856c admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:34.783926 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3190d0-a9f0-4234-a0e3-bb314ccdd506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:34.784674 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3190d0-a9f0-4234-a0e3-bb314ccdd506 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:34.785344 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3190d0-a9f0-4234-a0e3-bb314ccdd506 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:34.785692 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3190d0-a9f0-4234-a0e3-bb314ccdd506 admin admin] PATH_INFO: `/v3/projects/750d9c8d7db140deab48a6580625ffa7/users/2843cd5feab8472bbbc9ad0930be7176/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:34.788651 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3190d0-a9f0-4234-a0e3-bb314ccdd506 admin admin] RBAC: Authorizing `identity:create_grant(project_id=750d9c8d7db140deab48a6580625ffa7, user_id=2843cd5feab8472bbbc9ad0930be7176, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:34.791470 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3190d0-a9f0-4234-a0e3-bb314ccdd506 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:34.794250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:34.805428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1231/2473] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:39:34 2022] PUT /identity/v3/projects/750d9c8d7db140deab48a6580625ffa7/users/2843cd5feab8472bbbc9ad0930be7176/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:34.826282 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209b868-7dd1-47e0-8a02-4ceb9948ece2 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:34.826282 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209b868-7dd1-47e0-8a02-4ceb9948ece2 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:34.826282 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209b868-7dd1-47e0-8a02-4ceb9948ece2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:34.851155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-9209b868-7dd1-47e0-8a02-4ceb9948ece2 None None] MFA Rules not processed for user `2843cd5feab8472bbbc9ad0930be7176`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:34.852371 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-9209b868-7dd1-47e0-8a02-4ceb9948ece2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:34.937446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1243/2474] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:39:34 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:39:35.151518 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3190d0-a9f0-4234-a0e3-bb314ccdd506 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:35.232118 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3190d0-a9f0-4234-a0e3-bb314ccdd506 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:35.235823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a5eee9-26e1-4ad8-9412-e8620f1cb4a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:35.236430 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5eee9-26e1-4ad8-9412-e8620f1cb4a5 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:35.236704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5eee9-26e1-4ad8-9412-e8620f1cb4a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:35.236972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5eee9-26e1-4ad8-9412-e8620f1cb4a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:35.299446 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a5eee9-26e1-4ad8-9412-e8620f1cb4a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:35.300883 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a5eee9-26e1-4ad8-9412-e8620f1cb4a5 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:35.336944 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1232/2475] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:39:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:39:36.238146 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9209b868-7dd1-47e0-8a02-4ceb9948ece2 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:36.240482 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-9209b868-7dd1-47e0-8a02-4ceb9948ece2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:36.358595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9209b868-7dd1-47e0-8a02-4ceb9948ece2 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:36.381499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063fb56e-6941-4b7f-a7c4-08a9ba87271d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:36.382174 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fb56e-6941-4b7f-a7c4-08a9ba87271d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:36.382481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fb56e-6941-4b7f-a7c4-08a9ba87271d admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:36.382931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fb56e-6941-4b7f-a7c4-08a9ba87271d admin admin] PATH_INFO: `/v3/users/095a406c2a9e40e2a57b5eb298640276` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:36.398022 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063fb56e-6941-4b7f-a7c4-08a9ba87271d admin admin] RBAC: Authorizing `identity:delete_user(user_id=095a406c2a9e40e2a57b5eb298640276)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:36.404358 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063fb56e-6941-4b7f-a7c4-08a9ba87271d admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:36.657030 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:36.664976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-063fb56e-6941-4b7f-a7c4-08a9ba87271d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '095a406c2a9e40e2a57b5eb298640276'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:36.678638 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-063fb56e-6941-4b7f-a7c4-08a9ba87271d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '095a406c2a9e40e2a57b5eb298640276'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:36.687690 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-063fb56e-6941-4b7f-a7c4-08a9ba87271d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '095a406c2a9e40e2a57b5eb298640276'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:36.807449 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-063fb56e-6941-4b7f-a7c4-08a9ba87271d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '095a406c2a9e40e2a57b5eb298640276'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:36.808120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-063fb56e-6941-4b7f-a7c4-08a9ba87271d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '095a406c2a9e40e2a57b5eb298640276'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:36.817839 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1244/2476] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:39:36 2022] DELETE /identity/v3/users/095a406c2a9e40e2a57b5eb298640276 => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:37.392580 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a5eee9-26e1-4ad8-9412-e8620f1cb4a5 service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:37.404271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a5eee9-26e1-4ad8-9412-e8620f1cb4a5 service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:37.466445 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2cd4fb-1063-4417-846f-445a3fe2d2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:37.467107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2cd4fb-1063-4417-846f-445a3fe2d2fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:37.467381 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2cd4fb-1063-4417-846f-445a3fe2d2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:37.467648 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2cd4fb-1063-4417-846f-445a3fe2d2fb admin admin] PATH_INFO: `/v3/projects/39682f18e68946efb4a82c883a01dd5f` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:37.469203 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2cd4fb-1063-4417-846f-445a3fe2d2fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=39682f18e68946efb4a82c883a01dd5f)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:37.470671 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2cd4fb-1063-4417-846f-445a3fe2d2fb admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:37.601392 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5c2cd4fb-1063-4417-846f-445a3fe2d2fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39682f18e68946efb4a82c883a01dd5f', 'request_id': 'req-5c2cd4fb-1063-4417-846f-445a3fe2d2fb'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:37.602534 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:INSERT=1,keystone:SELECT=95,keystone:DELETE=4 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:37.608999 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5c2cd4fb-1063-4417-846f-445a3fe2d2fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39682f18e68946efb4a82c883a01dd5f', 'request_id': 'req-5c2cd4fb-1063-4417-846f-445a3fe2d2fb'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:37.614517 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5c2cd4fb-1063-4417-846f-445a3fe2d2fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39682f18e68946efb4a82c883a01dd5f', 'request_id': 'req-5c2cd4fb-1063-4417-846f-445a3fe2d2fb'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:37.699609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5c2cd4fb-1063-4417-846f-445a3fe2d2fb admin admin] The token cache is being invalidate because project 39682f18e68946efb4a82c883a01dd5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:39:37.699902 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5c2cd4fb-1063-4417-846f-445a3fe2d2fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:37.700671 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5c2cd4fb-1063-4417-846f-445a3fe2d2fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:37.703361 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1233/2477] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:39:37 2022] DELETE /identity/v3/projects/39682f18e68946efb4a82c883a01dd5f => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:39:37.706406 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063fb56e-6941-4b7f-a7c4-08a9ba87271d admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:37.707875 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-063fb56e-6941-4b7f-a7c4-08a9ba87271d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:37.722253 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063fb56e-6941-4b7f-a7c4-08a9ba87271d admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:37.781607 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b97bd-0400-436b-8ef5-b6658dd02c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:37.782233 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b97bd-0400-436b-8ef5-b6658dd02c74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:37.782544 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b97bd-0400-436b-8ef5-b6658dd02c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:37.782820 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b97bd-0400-436b-8ef5-b6658dd02c74 admin admin] PATH_INFO: `/v3/users/e5746817cf1c46ea80ca1c6157b034c8` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:37.784101 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b97bd-0400-436b-8ef5-b6658dd02c74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5746817cf1c46ea80ca1c6157b034c8)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:37.786060 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b97bd-0400-436b-8ef5-b6658dd02c74 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:37.969015 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-f76b97bd-0400-436b-8ef5-b6658dd02c74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5746817cf1c46ea80ca1c6157b034c8'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:37.981567 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-f76b97bd-0400-436b-8ef5-b6658dd02c74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5746817cf1c46ea80ca1c6157b034c8'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:37.988557 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-f76b97bd-0400-436b-8ef5-b6658dd02c74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5746817cf1c46ea80ca1c6157b034c8'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:38.069193 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-f76b97bd-0400-436b-8ef5-b6658dd02c74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5746817cf1c46ea80ca1c6157b034c8'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:38.069809 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-f76b97bd-0400-436b-8ef5-b6658dd02c74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5746817cf1c46ea80ca1c6157b034c8'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:38.076888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1245/2478] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:39:37 2022] DELETE /identity/v3/users/e5746817cf1c46ea80ca1c6157b034c8 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:38.318570 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2cd4fb-1063-4417-846f-445a3fe2d2fb admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:38.320294 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-5c2cd4fb-1063-4417-846f-445a3fe2d2fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:38.335251 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2cd4fb-1063-4417-846f-445a3fe2d2fb admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:38.397634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86088963-38b8-4f93-8244-cea09da8da49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:38.398311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86088963-38b8-4f93-8244-cea09da8da49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:38.398576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86088963-38b8-4f93-8244-cea09da8da49 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:38.398851 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86088963-38b8-4f93-8244-cea09da8da49 admin admin] PATH_INFO: `/v3/projects/2d77dc6509724be7ae2c30224a34622b` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:38.400174 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86088963-38b8-4f93-8244-cea09da8da49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d77dc6509724be7ae2c30224a34622b)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:38.402093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86088963-38b8-4f93-8244-cea09da8da49 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:38.505996 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-86088963-38b8-4f93-8244-cea09da8da49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d77dc6509724be7ae2c30224a34622b', 'request_id': 'req-86088963-38b8-4f93-8244-cea09da8da49'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:38.513062 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-86088963-38b8-4f93-8244-cea09da8da49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d77dc6509724be7ae2c30224a34622b', 'request_id': 'req-86088963-38b8-4f93-8244-cea09da8da49'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:38.518789 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-86088963-38b8-4f93-8244-cea09da8da49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d77dc6509724be7ae2c30224a34622b', 'request_id': 'req-86088963-38b8-4f93-8244-cea09da8da49'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:38.597218 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-86088963-38b8-4f93-8244-cea09da8da49 admin admin] The token cache is being invalidate because project 2d77dc6509724be7ae2c30224a34622b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:39:38.597654 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-86088963-38b8-4f93-8244-cea09da8da49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:38.598313 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-86088963-38b8-4f93-8244-cea09da8da49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:38.599888 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1234/2479] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:39:38 2022] DELETE /identity/v3/projects/2d77dc6509724be7ae2c30224a34622b => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:47.438911 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b97bd-0400-436b-8ef5-b6658dd02c74 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:47.440412 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-f76b97bd-0400-436b-8ef5-b6658dd02c74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:47.484955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:47.500996 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b97bd-0400-436b-8ef5-b6658dd02c74 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:47.548676 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61d6fe-d386-47c2-a20d-25ffa93e26a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:47.549287 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61d6fe-d386-47c2-a20d-25ffa93e26a5 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:47.549562 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61d6fe-d386-47c2-a20d-25ffa93e26a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:47.549830 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61d6fe-d386-47c2-a20d-25ffa93e26a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:47.551589 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-be61d6fe-d386-47c2-a20d-25ffa93e26a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:47.615025 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61d6fe-d386-47c2-a20d-25ffa93e26a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:47.616555 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61d6fe-d386-47c2-a20d-25ffa93e26a5 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:47.626460 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1246/2480] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:39:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:39:48.593600 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=57,keystone:DELETE=10,keystone:INSERT=2 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:49.551609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86088963-38b8-4f93-8244-cea09da8da49 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:49.553110 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-86088963-38b8-4f93-8244-cea09da8da49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:49.565236 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86088963-38b8-4f93-8244-cea09da8da49 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:49.568946 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ede38-d435-40eb-a0af-0f85ec65e412 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:49.569540 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ede38-d435-40eb-a0af-0f85ec65e412 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:49.569820 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ede38-d435-40eb-a0af-0f85ec65e412 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:49.570093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ede38-d435-40eb-a0af-0f85ec65e412 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:49.575466 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-e13ede38-d435-40eb-a0af-0f85ec65e412 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:49.638920 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61d6fe-d386-47c2-a20d-25ffa93e26a5 service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:49.640447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-be61d6fe-d386-47c2-a20d-25ffa93e26a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:49.646091 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ede38-d435-40eb-a0af-0f85ec65e412 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:49.647666 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ede38-d435-40eb-a0af-0f85ec65e412 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:49.660168 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1235/2481] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:39:49 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:39:49.750190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61d6fe-d386-47c2-a20d-25ffa93e26a5 service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:49.756407 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912c576-7088-4c98-b171-f226ff211aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:49.757052 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912c576-7088-4c98-b171-f226ff211aea admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:49.757323 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912c576-7088-4c98-b171-f226ff211aea admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:49.757656 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912c576-7088-4c98-b171-f226ff211aea admin admin] PATH_INFO: `/v3/users/3156139a06844f329f4820d054966543` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:49.759390 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2912c576-7088-4c98-b171-f226ff211aea admin admin] RBAC: Authorizing `identity:delete_user(user_id=3156139a06844f329f4820d054966543)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:49.760838 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2912c576-7088-4c98-b171-f226ff211aea admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:50.002642 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-2912c576-7088-4c98-b171-f226ff211aea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3156139a06844f329f4820d054966543'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:50.007624 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:50.011952 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-2912c576-7088-4c98-b171-f226ff211aea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3156139a06844f329f4820d054966543'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:50.018826 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-2912c576-7088-4c98-b171-f226ff211aea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3156139a06844f329f4820d054966543'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:50.116746 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-2912c576-7088-4c98-b171-f226ff211aea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3156139a06844f329f4820d054966543'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:50.117440 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-2912c576-7088-4c98-b171-f226ff211aea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3156139a06844f329f4820d054966543'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:50.124129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1247/2482] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:39:49 2022] DELETE /identity/v3/users/3156139a06844f329f4820d054966543 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:39:50.128566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fc49b-04a9-4540-a8b4-c90076cd6792 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:50.128850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fc49b-04a9-4540-a8b4-c90076cd6792 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:50.129123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fc49b-04a9-4540-a8b4-c90076cd6792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:50.158132 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-9e0fc49b-04a9-4540-a8b4-c90076cd6792 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:50.159817 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-9e0fc49b-04a9-4540-a8b4-c90076cd6792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:50.224540 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1236/2483] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:39:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:39:50.235470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912c576-7088-4c98-b171-f226ff211aea admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:50.237709 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2912c576-7088-4c98-b171-f226ff211aea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:50.303863 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912c576-7088-4c98-b171-f226ff211aea admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:50.308226 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b93c96-aa4e-4bdd-90ae-81036098eb99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:50.308967 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b93c96-aa4e-4bdd-90ae-81036098eb99 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:50.309861 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b93c96-aa4e-4bdd-90ae-81036098eb99 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:50.310283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b93c96-aa4e-4bdd-90ae-81036098eb99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:50.319994 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b93c96-aa4e-4bdd-90ae-81036098eb99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:50.321647 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b93c96-aa4e-4bdd-90ae-81036098eb99 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:50.332285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1248/2484] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:39:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:39:50.969262 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0fc49b-04a9-4540-a8b4-c90076cd6792 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:50.971346 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-9e0fc49b-04a9-4540-a8b4-c90076cd6792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:50.989409 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0fc49b-04a9-4540-a8b4-c90076cd6792 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:51.001087 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d4211-c104-4fa4-84b4-2fcce2065d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:51.001970 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d4211-c104-4fa4-84b4-2fcce2065d1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:51.002479 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d4211-c104-4fa4-84b4-2fcce2065d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:51.002897 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d4211-c104-4fa4-84b4-2fcce2065d1a admin admin] PATH_INFO: `/v3/projects/87226efbc8db4654b947e386737a1ccd` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:51.005299 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d4211-c104-4fa4-84b4-2fcce2065d1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=87226efbc8db4654b947e386737a1ccd)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:51.008235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d4211-c104-4fa4-84b4-2fcce2065d1a admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:51.175891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-f50d4211-c104-4fa4-84b4-2fcce2065d1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87226efbc8db4654b947e386737a1ccd', 'request_id': 'req-f50d4211-c104-4fa4-84b4-2fcce2065d1a'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:51.185001 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-f50d4211-c104-4fa4-84b4-2fcce2065d1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87226efbc8db4654b947e386737a1ccd', 'request_id': 'req-f50d4211-c104-4fa4-84b4-2fcce2065d1a'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:51.192065 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-f50d4211-c104-4fa4-84b4-2fcce2065d1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87226efbc8db4654b947e386737a1ccd', 'request_id': 'req-f50d4211-c104-4fa4-84b4-2fcce2065d1a'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:51.306978 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-f50d4211-c104-4fa4-84b4-2fcce2065d1a admin admin] The token cache is being invalidate because project 87226efbc8db4654b947e386737a1ccd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:39:51.306978 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-f50d4211-c104-4fa4-84b4-2fcce2065d1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:51.306978 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-f50d4211-c104-4fa4-84b4-2fcce2065d1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:51.308867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1237/2485] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:39:50 2022] DELETE /identity/v3/projects/87226efbc8db4654b947e386737a1ccd => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:51.961480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b93c96-aa4e-4bdd-90ae-81036098eb99 service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:51.962993 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-b5b93c96-aa4e-4bdd-90ae-81036098eb99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:51.977380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b93c96-aa4e-4bdd-90ae-81036098eb99 service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:52.041247 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897eb45-233b-4b43-98c7-6448fdb1777f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:52.042443 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897eb45-233b-4b43-98c7-6448fdb1777f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:52.042724 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897eb45-233b-4b43-98c7-6448fdb1777f admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:52.042988 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897eb45-233b-4b43-98c7-6448fdb1777f admin admin] PATH_INFO: `/v3/users/35dae86d2bd24a3a9b27489422552d26` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:52.044232 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c897eb45-233b-4b43-98c7-6448fdb1777f admin admin] RBAC: Authorizing `identity:delete_user(user_id=35dae86d2bd24a3a9b27489422552d26)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:52.046146 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c897eb45-233b-4b43-98c7-6448fdb1777f admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:52.279778 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c897eb45-233b-4b43-98c7-6448fdb1777f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35dae86d2bd24a3a9b27489422552d26'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:52.300736 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c897eb45-233b-4b43-98c7-6448fdb1777f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35dae86d2bd24a3a9b27489422552d26'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:52.308477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:52.313249 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c897eb45-233b-4b43-98c7-6448fdb1777f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35dae86d2bd24a3a9b27489422552d26'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:52.430825 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c897eb45-233b-4b43-98c7-6448fdb1777f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35dae86d2bd24a3a9b27489422552d26'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:52.431404 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c897eb45-233b-4b43-98c7-6448fdb1777f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35dae86d2bd24a3a9b27489422552d26'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:52.457492 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1249/2486] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:39:51 2022] DELETE /identity/v3/users/35dae86d2bd24a3a9b27489422552d26 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:52.461125 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e9fd6-2d56-4172-b7a5-13e3a90b6d66 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:52.461629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e9fd6-2d56-4172-b7a5-13e3a90b6d66 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:52.461716 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e9fd6-2d56-4172-b7a5-13e3a90b6d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:52.488001 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-5a3e9fd6-2d56-4172-b7a5-13e3a90b6d66 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:52.489418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-5a3e9fd6-2d56-4172-b7a5-13e3a90b6d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:52.495947 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:52.556599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1238/2487] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:39:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:39:52.567071 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897eb45-233b-4b43-98c7-6448fdb1777f admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:52.568766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-c897eb45-233b-4b43-98c7-6448fdb1777f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:52.649892 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897eb45-233b-4b43-98c7-6448fdb1777f admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:52.656030 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d671e-338d-415e-b37f-f32edfcac910 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:52.656676 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d671e-338d-415e-b37f-f32edfcac910 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:52.656930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d671e-338d-415e-b37f-f32edfcac910 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:52.657196 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d671e-338d-415e-b37f-f32edfcac910 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:52.667733 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339d671e-338d-415e-b37f-f32edfcac910 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:52.669443 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339d671e-338d-415e-b37f-f32edfcac910 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:52.687490 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1250/2488] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:39:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:39:53.205158 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e9fd6-2d56-4172-b7a5-13e3a90b6d66 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:53.206714 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-5a3e9fd6-2d56-4172-b7a5-13e3a90b6d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:53.219793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e9fd6-2d56-4172-b7a5-13e3a90b6d66 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:53.225361 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb944a43-4a1f-4bec-a26d-fc93771a46b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:53.226184 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb944a43-4a1f-4bec-a26d-fc93771a46b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:53.226603 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb944a43-4a1f-4bec-a26d-fc93771a46b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:53.226935 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb944a43-4a1f-4bec-a26d-fc93771a46b9 admin admin] PATH_INFO: `/v3/projects/44f6fc01500b4750a9b3882424813c13` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:53.228208 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb944a43-4a1f-4bec-a26d-fc93771a46b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44f6fc01500b4750a9b3882424813c13)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:53.230259 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb944a43-4a1f-4bec-a26d-fc93771a46b9 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:53.583552 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-fb944a43-4a1f-4bec-a26d-fc93771a46b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44f6fc01500b4750a9b3882424813c13', 'request_id': 'req-fb944a43-4a1f-4bec-a26d-fc93771a46b9'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:53.595506 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-fb944a43-4a1f-4bec-a26d-fc93771a46b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44f6fc01500b4750a9b3882424813c13', 'request_id': 'req-fb944a43-4a1f-4bec-a26d-fc93771a46b9'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:53.605327 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-fb944a43-4a1f-4bec-a26d-fc93771a46b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44f6fc01500b4750a9b3882424813c13', 'request_id': 'req-fb944a43-4a1f-4bec-a26d-fc93771a46b9'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:53.785104 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-fb944a43-4a1f-4bec-a26d-fc93771a46b9 admin admin] The token cache is being invalidate because project 44f6fc01500b4750a9b3882424813c13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:39:53.785402 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-fb944a43-4a1f-4bec-a26d-fc93771a46b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:53.786079 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-fb944a43-4a1f-4bec-a26d-fc93771a46b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:53.787601 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1239/2489] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:39:53 2022] DELETE /identity/v3/projects/44f6fc01500b4750a9b3882424813c13 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:39:53.791568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d671e-338d-415e-b37f-f32edfcac910 service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:53.802503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d671e-338d-415e-b37f-f32edfcac910 service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:53.896698 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03a460-7c4a-4250-8c2b-8423a11c861c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:53.897737 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03a460-7c4a-4250-8c2b-8423a11c861c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:53.898182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03a460-7c4a-4250-8c2b-8423a11c861c admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:53.898626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03a460-7c4a-4250-8c2b-8423a11c861c admin admin] PATH_INFO: `/v3/users/fcd4b6da197d4717a6cd691f53023058` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:53.900059 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d03a460-7c4a-4250-8c2b-8423a11c861c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcd4b6da197d4717a6cd691f53023058)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:53.903144 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d03a460-7c4a-4250-8c2b-8423a11c861c admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:54.142205 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-0d03a460-7c4a-4250-8c2b-8423a11c861c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcd4b6da197d4717a6cd691f53023058'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:54.161258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-0d03a460-7c4a-4250-8c2b-8423a11c861c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcd4b6da197d4717a6cd691f53023058'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:54.389099 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-0d03a460-7c4a-4250-8c2b-8423a11c861c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcd4b6da197d4717a6cd691f53023058'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:54.652519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-0d03a460-7c4a-4250-8c2b-8423a11c861c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcd4b6da197d4717a6cd691f53023058'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:54.652519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-0d03a460-7c4a-4250-8c2b-8423a11c861c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcd4b6da197d4717a6cd691f53023058'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:54.652519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:54.655875 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1251/2490] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:39:53 2022] DELETE /identity/v3/users/fcd4b6da197d4717a6cd691f53023058 => generated 0 bytes in 865 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:55.142689 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb944a43-4a1f-4bec-a26d-fc93771a46b9 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:55.143820 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-fb944a43-4a1f-4bec-a26d-fc93771a46b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:55.176289 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb944a43-4a1f-4bec-a26d-fc93771a46b9 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:55.255459 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c726c1-f423-4aaf-a52d-0025e9a3b7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:55.256093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c726c1-f423-4aaf-a52d-0025e9a3b7e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:55.256424 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c726c1-f423-4aaf-a52d-0025e9a3b7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:55.256679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c726c1-f423-4aaf-a52d-0025e9a3b7e1 admin admin] PATH_INFO: `/v3/projects/f9ac4c1e5fc641049be06afa40b4c6ea` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:55.257835 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c726c1-f423-4aaf-a52d-0025e9a3b7e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9ac4c1e5fc641049be06afa40b4c6ea)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:55.259842 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c726c1-f423-4aaf-a52d-0025e9a3b7e1 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:55.382485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:55.391105 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-29c726c1-f423-4aaf-a52d-0025e9a3b7e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9ac4c1e5fc641049be06afa40b4c6ea', 'request_id': 'req-29c726c1-f423-4aaf-a52d-0025e9a3b7e1'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:55.397251 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-29c726c1-f423-4aaf-a52d-0025e9a3b7e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9ac4c1e5fc641049be06afa40b4c6ea', 'request_id': 'req-29c726c1-f423-4aaf-a52d-0025e9a3b7e1'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:55.403295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-29c726c1-f423-4aaf-a52d-0025e9a3b7e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9ac4c1e5fc641049be06afa40b4c6ea', 'request_id': 'req-29c726c1-f423-4aaf-a52d-0025e9a3b7e1'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:55.472582 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-29c726c1-f423-4aaf-a52d-0025e9a3b7e1 admin admin] The token cache is being invalidate because project f9ac4c1e5fc641049be06afa40b4c6ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:39:55.472905 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-29c726c1-f423-4aaf-a52d-0025e9a3b7e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:55.473886 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-29c726c1-f423-4aaf-a52d-0025e9a3b7e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:55.475588 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1240/2491] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:39:55 2022] DELETE /identity/v3/projects/f9ac4c1e5fc641049be06afa40b4c6ea => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:55.481482 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0957159-4a9c-46e2-b834-134a9ee05459 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:55.487054 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0957159-4a9c-46e2-b834-134a9ee05459 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:55.487054 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0957159-4a9c-46e2-b834-134a9ee05459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:55.535440 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-e0957159-4a9c-46e2-b834-134a9ee05459 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:55.536695 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-e0957159-4a9c-46e2-b834-134a9ee05459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:55.602557 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1252/2492] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:39:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:39:55.612162 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b3900-831f-4b90-98df-85549c9b96e2 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:55.612445 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b3900-831f-4b90-98df-85549c9b96e2 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:55.612893 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b3900-831f-4b90-98df-85549c9b96e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:55.638482 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-d02b3900-831f-4b90-98df-85549c9b96e2 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:55.639782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-d02b3900-831f-4b90-98df-85549c9b96e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:55.648825 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1241/2493] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:39:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:39:55.654014 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0957159-4a9c-46e2-b834-134a9ee05459 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:55.664647 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0957159-4a9c-46e2-b834-134a9ee05459 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:55.669852 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf6b807-f853-4b8c-b173-4d5e474922ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:55.670462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6b807-f853-4b8c-b173-4d5e474922ab admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:55.670751 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6b807-f853-4b8c-b173-4d5e474922ab admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:55.671074 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6b807-f853-4b8c-b173-4d5e474922ab admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:55.671694 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf6b807-f853-4b8c-b173-4d5e474922ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:39:55.672000 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf6b807-f853-4b8c-b173-4d5e474922ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:55.673674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf6b807-f853-4b8c-b173-4d5e474922ab admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:55.687907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1253/2494] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:39:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:39:55.692415 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b3900-831f-4b90-98df-85549c9b96e2 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:55.703470 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b3900-831f-4b90-98df-85549c9b96e2 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:55.708977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d2b4a-7688-4a14-a4a8-83478b802187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:55.709809 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d2b4a-7688-4a14-a4a8-83478b802187 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:55.710071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d2b4a-7688-4a14-a4a8-83478b802187 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:55.710346 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d2b4a-7688-4a14-a4a8-83478b802187 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:55.711303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9d2b4a-7688-4a14-a4a8-83478b802187 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:55.721488 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9d2b4a-7688-4a14-a4a8-83478b802187 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:55.858342 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1242/2495] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:39:55 2022] POST /identity/v3/projects => generated 387 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:39:55.862555 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf6b807-f853-4b8c-b173-4d5e474922ab admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:55.874891 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf6b807-f853-4b8c-b173-4d5e474922ab admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:55.937604 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09992c10-b884-4c4f-80d9-b1d3445c56c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:55.939011 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09992c10-b884-4c4f-80d9-b1d3445c56c1 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:55.939011 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09992c10-b884-4c4f-80d9-b1d3445c56c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:55.939011 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09992c10-b884-4c4f-80d9-b1d3445c56c1 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:55.939398 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09992c10-b884-4c4f-80d9-b1d3445c56c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:55.941318 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09992c10-b884-4c4f-80d9-b1d3445c56c1 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:56.043175 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1254/2496] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:39:55 2022] POST /identity/v3/users => generated 294 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:39:56.047521 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d2b4a-7688-4a14-a4a8-83478b802187 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:56.058299 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d2b4a-7688-4a14-a4a8-83478b802187 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:56.064435 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb179d-f442-4017-b256-54c87194a771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:56.064877 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb179d-f442-4017-b256-54c87194a771 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:56.065148 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb179d-f442-4017-b256-54c87194a771 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:56.065411 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb179d-f442-4017-b256-54c87194a771 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:56.066046 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fb179d-f442-4017-b256-54c87194a771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:56.067689 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fb179d-f442-4017-b256-54c87194a771 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:56.079008 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1243/2497] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:39:56.083433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09992c10-b884-4c4f-80d9-b1d3445c56c1 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:56.085501 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb179d-f442-4017-b256-54c87194a771 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:56.087546 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-f6fb179d-f442-4017-b256-54c87194a771 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:56.094981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09992c10-b884-4c4f-80d9-b1d3445c56c1 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:56.099959 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6409c21-de07-42a2-b57b-78d3206fcf1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:56.100662 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6409c21-de07-42a2-b57b-78d3206fcf1a admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:56.101036 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6409c21-de07-42a2-b57b-78d3206fcf1a admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:56.101399 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6409c21-de07-42a2-b57b-78d3206fcf1a admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:56.101911 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb179d-f442-4017-b256-54c87194a771 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:56.102229 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6409c21-de07-42a2-b57b-78d3206fcf1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:56.103979 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6409c21-de07-42a2-b57b-78d3206fcf1a admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:56.107636 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0987a4a4-4d2e-4f77-aaf9-dc13e29849fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:56.108274 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987a4a4-4d2e-4f77-aaf9-dc13e29849fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:56.108561 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987a4a4-4d2e-4f77-aaf9-dc13e29849fa admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:56.108907 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987a4a4-4d2e-4f77-aaf9-dc13e29849fa admin admin] PATH_INFO: `/v3/users/f177a8a49f274929a40ffeb7a551c003` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:56.110033 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0987a4a4-4d2e-4f77-aaf9-dc13e29849fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=f177a8a49f274929a40ffeb7a551c003)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:56.112010 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0987a4a4-4d2e-4f77-aaf9-dc13e29849fa admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:56.116436 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1255/2498] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:39:56.120537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6409c21-de07-42a2-b57b-78d3206fcf1a admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:56.130970 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6409c21-de07-42a2-b57b-78d3206fcf1a admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:56.137506 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46d5f2e-7aa1-45f0-b925-a1a01d982124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:56.138311 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d5f2e-7aa1-45f0-b925-a1a01d982124 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:56.138618 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d5f2e-7aa1-45f0-b925-a1a01d982124 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:56.139001 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d5f2e-7aa1-45f0-b925-a1a01d982124 admin admin] PATH_INFO: `/v3/projects/446c6258a94d48bc9d95e29693d79c2d/users/97eff25204014b698b82c51b6b82f7f5/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:56.156740 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46d5f2e-7aa1-45f0-b925-a1a01d982124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=446c6258a94d48bc9d95e29693d79c2d, user_id=97eff25204014b698b82c51b6b82f7f5, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:56.159454 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46d5f2e-7aa1-45f0-b925-a1a01d982124 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:56.317492 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1256/2499] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:39:56 2022] PUT /identity/v3/projects/446c6258a94d48bc9d95e29693d79c2d/users/97eff25204014b698b82c51b6b82f7f5/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:39:56.328685 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d589a-6494-4b0a-af65-372fad3e427c None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:56.328973 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d589a-6494-4b0a-af65-372fad3e427c None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:56.329295 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d589a-6494-4b0a-af65-372fad3e427c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:56.363333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-c41d589a-6494-4b0a-af65-372fad3e427c None None] MFA Rules not processed for user `97eff25204014b698b82c51b6b82f7f5`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:56.364626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-c41d589a-6494-4b0a-af65-372fad3e427c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:56.373042 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:56.399053 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-0987a4a4-4d2e-4f77-aaf9-dc13e29849fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f177a8a49f274929a40ffeb7a551c003'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:56.442591 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1257/2500] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:39:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:39:56.444063 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f125b80-391e-4d46-aa79-aa6f7f5f62bd None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:56.444339 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f125b80-391e-4d46-aa79-aa6f7f5f62bd None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:56.445484 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f125b80-391e-4d46-aa79-aa6f7f5f62bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:56.474814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-0f125b80-391e-4d46-aa79-aa6f7f5f62bd None None] MFA Rules not processed for user `d5d0010f584249b4a22fd31634c9fd8b`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:56.476120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-0f125b80-391e-4d46-aa79-aa6f7f5f62bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:56.504970 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-0987a4a4-4d2e-4f77-aaf9-dc13e29849fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f177a8a49f274929a40ffeb7a551c003'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:56.505806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-0987a4a4-4d2e-4f77-aaf9-dc13e29849fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f177a8a49f274929a40ffeb7a551c003'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:56.506454 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-0987a4a4-4d2e-4f77-aaf9-dc13e29849fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f177a8a49f274929a40ffeb7a551c003'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:56.523449 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-0987a4a4-4d2e-4f77-aaf9-dc13e29849fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f177a8a49f274929a40ffeb7a551c003'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:56.534204 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1258/2501] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:39:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:39:56.535213 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f125b80-391e-4d46-aa79-aa6f7f5f62bd None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:56.537485 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-0f125b80-391e-4d46-aa79-aa6f7f5f62bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:56.545908 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1244/2502] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:39:56 2022] DELETE /identity/v3/users/f177a8a49f274929a40ffeb7a551c003 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:39:56.550443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5c9d4-394a-442b-8df4-d7fc2dcf4274 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:56.550734 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5c9d4-394a-442b-8df4-d7fc2dcf4274 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:56.556843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5c9d4-394a-442b-8df4-d7fc2dcf4274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:56.574659 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-c5b5c9d4-394a-442b-8df4-d7fc2dcf4274 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:56.575871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-c5b5c9d4-394a-442b-8df4-d7fc2dcf4274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:56.632418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:56.638423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f125b80-391e-4d46-aa79-aa6f7f5f62bd None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:56.648917 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1245/2503] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:39:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:39:56.672799 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5c9d4-394a-442b-8df4-d7fc2dcf4274 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:56.676155 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-c5b5c9d4-394a-442b-8df4-d7fc2dcf4274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:56.714378 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b97a80-036a-49bf-b53f-433be3615e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:56.715229 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b97a80-036a-49bf-b53f-433be3615e92 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:56.715706 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b97a80-036a-49bf-b53f-433be3615e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:56.716242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b97a80-036a-49bf-b53f-433be3615e92 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:56.716888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b97a80-036a-49bf-b53f-433be3615e92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:56.718912 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b97a80-036a-49bf-b53f-433be3615e92 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:56.742151 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5c9d4-394a-442b-8df4-d7fc2dcf4274 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:56.745853 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc6f58b-1ec8-4bbb-a16d-b21807baba33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:56.747023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6f58b-1ec8-4bbb-a16d-b21807baba33 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:56.747296 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6f58b-1ec8-4bbb-a16d-b21807baba33 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:56.747566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6f58b-1ec8-4bbb-a16d-b21807baba33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:56.757803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc6f58b-1ec8-4bbb-a16d-b21807baba33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:56.759261 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc6f58b-1ec8-4bbb-a16d-b21807baba33 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:56.769999 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1246/2504] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:39:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:39:56.803936 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1259/2505] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:39:56 2022] POST /identity/v3/projects => generated 387 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:39:56.808167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc6f58b-1ec8-4bbb-a16d-b21807baba33 service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:56.818602 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc6f58b-1ec8-4bbb-a16d-b21807baba33 service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:56.883773 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321d0ce-0ee9-4416-b6a9-327501471165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:56.884387 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321d0ce-0ee9-4416-b6a9-327501471165 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:56.884661 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321d0ce-0ee9-4416-b6a9-327501471165 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:56.884924 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321d0ce-0ee9-4416-b6a9-327501471165 admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:56.885815 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321d0ce-0ee9-4416-b6a9-327501471165 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:56.887721 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321d0ce-0ee9-4416-b6a9-327501471165 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:56.921009 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:56.953602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b97a80-036a-49bf-b53f-433be3615e92 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:56.982828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1247/2506] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:39:56 2022] POST /identity/v3/users => generated 294 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:39:56.985546 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321d0ce-0ee9-4416-b6a9-327501471165 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:56.998467 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321d0ce-0ee9-4416-b6a9-327501471165 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:57.006625 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f97d6-1a44-4a93-8634-8e7b5faaed5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:57.006625 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f97d6-1a44-4a93-8634-8e7b5faaed5d admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:57.006625 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f97d6-1a44-4a93-8634-8e7b5faaed5d admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:57.006625 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f97d6-1a44-4a93-8634-8e7b5faaed5d admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:57.006918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f97d6-1a44-4a93-8634-8e7b5faaed5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:57.008610 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f97d6-1a44-4a93-8634-8e7b5faaed5d admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:57.022020 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1248/2507] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:39:57.025985 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f97d6-1a44-4a93-8634-8e7b5faaed5d admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:57.037865 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f97d6-1a44-4a93-8634-8e7b5faaed5d admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:57.054438 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b97a80-036a-49bf-b53f-433be3615e92 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:57.058337 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56bedd-8a43-458f-aef9-304828d96407 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:57.060389 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56bedd-8a43-458f-aef9-304828d96407 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:57.060709 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56bedd-8a43-458f-aef9-304828d96407 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:57.060973 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56bedd-8a43-458f-aef9-304828d96407 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:57.062468 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f2c6c-70d9-44d4-9570-ee8df2733170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:57.063950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f2c6c-70d9-44d4-9570-ee8df2733170 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:57.063950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f2c6c-70d9-44d4-9570-ee8df2733170 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:57.063950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f2c6c-70d9-44d4-9570-ee8df2733170 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:57.065982 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-1c56bedd-8a43-458f-aef9-304828d96407 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:57.069079 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2f2c6c-70d9-44d4-9570-ee8df2733170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:57.070910 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2f2c6c-70d9-44d4-9570-ee8df2733170 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:57.078083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c56bedd-8a43-458f-aef9-304828d96407 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:57.079941 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c56bedd-8a43-458f-aef9-304828d96407 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:57.093657 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1249/2508] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:39:57.094017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1260/2509] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:39:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:39:57.101438 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f2c6c-70d9-44d4-9570-ee8df2733170 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:57.128048 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f2c6c-70d9-44d4-9570-ee8df2733170 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:57.132862 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d9f4d-2b50-452f-a69a-246a5d77538f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:57.133588 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d9f4d-2b50-452f-a69a-246a5d77538f admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:57.133816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d9f4d-2b50-452f-a69a-246a5d77538f admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:57.134086 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d9f4d-2b50-452f-a69a-246a5d77538f admin admin] PATH_INFO: `/v3/projects/7ee1405efd564d8fb226e5fbb2a2ae41/users/5801069cc5d54a47aa6fbd7001f4cf6a/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:57.152787 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d9f4d-2b50-452f-a69a-246a5d77538f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ee1405efd564d8fb226e5fbb2a2ae41, user_id=5801069cc5d54a47aa6fbd7001f4cf6a, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:57.156306 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d9f4d-2b50-452f-a69a-246a5d77538f admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:57.288350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1250/2510] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:39:57 2022] PUT /identity/v3/projects/7ee1405efd564d8fb226e5fbb2a2ae41/users/5801069cc5d54a47aa6fbd7001f4cf6a/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:57.299901 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915af661-6817-4368-bf15-80285f7e29d4 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:57.300217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915af661-6817-4368-bf15-80285f7e29d4 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:57.300494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915af661-6817-4368-bf15-80285f7e29d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:57.337890 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-915af661-6817-4368-bf15-80285f7e29d4 None None] MFA Rules not processed for user `5801069cc5d54a47aa6fbd7001f4cf6a`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:57.340204 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-915af661-6817-4368-bf15-80285f7e29d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:57.415516 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d9f4d-2b50-452f-a69a-246a5d77538f admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:57.417491 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-137d9f4d-2b50-452f-a69a-246a5d77538f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:57.417790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1261/2511] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:39:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:39:57.418785 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915af661-6817-4368-bf15-80285f7e29d4 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:57.477276 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:57.491605 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915af661-6817-4368-bf15-80285f7e29d4 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:57.496642 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a165d-adf2-4123-a366-a2de8b3b7f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:57.497397 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a165d-adf2-4123-a366-a2de8b3b7f97 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:57.497556 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a165d-adf2-4123-a366-a2de8b3b7f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:57.497766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a165d-adf2-4123-a366-a2de8b3b7f97 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:57.498440 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a165d-adf2-4123-a366-a2de8b3b7f97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:57.500330 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a165d-adf2-4123-a366-a2de8b3b7f97 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:57.507123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d9f4d-2b50-452f-a69a-246a5d77538f admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:57.514075 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf5e85-45db-49ae-9038-421724a91f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:57.517227 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf5e85-45db-49ae-9038-421724a91f64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:57.517227 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf5e85-45db-49ae-9038-421724a91f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:57.517227 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf5e85-45db-49ae-9038-421724a91f64 admin admin] PATH_INFO: `/v3/projects/a50199051ad04d44aaa79918fa839beb` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:57.517227 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf5e85-45db-49ae-9038-421724a91f64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a50199051ad04d44aaa79918fa839beb)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:57.517641 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:57.519189 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf5e85-45db-49ae-9038-421724a91f64 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:57.638057 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1262/2512] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:39:57 2022] POST /identity/v3/projects => generated 385 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:39:57.645497 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a165d-adf2-4123-a366-a2de8b3b7f97 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:57.708875 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a165d-adf2-4123-a366-a2de8b3b7f97 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:57.713537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240be56b-898e-4b8d-ab6d-211ab66b4ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:57.714133 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240be56b-898e-4b8d-ab6d-211ab66b4ee4 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:57.714433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240be56b-898e-4b8d-ab6d-211ab66b4ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:57.714707 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240be56b-898e-4b8d-ab6d-211ab66b4ee4 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:57.715337 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240be56b-898e-4b8d-ab6d-211ab66b4ee4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:57.717544 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240be56b-898e-4b8d-ab6d-211ab66b4ee4 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:57.739206 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-f4cf5e85-45db-49ae-9038-421724a91f64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a50199051ad04d44aaa79918fa839beb', 'request_id': 'req-f4cf5e85-45db-49ae-9038-421724a91f64'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:57.745381 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-f4cf5e85-45db-49ae-9038-421724a91f64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a50199051ad04d44aaa79918fa839beb', 'request_id': 'req-f4cf5e85-45db-49ae-9038-421724a91f64'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:57.751366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-f4cf5e85-45db-49ae-9038-421724a91f64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a50199051ad04d44aaa79918fa839beb', 'request_id': 'req-f4cf5e85-45db-49ae-9038-421724a91f64'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:57.893937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1263/2513] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:39:57 2022] POST /identity/v3/users => generated 299 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:39:57.897875 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240be56b-898e-4b8d-ab6d-211ab66b4ee4 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:57.907908 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240be56b-898e-4b8d-ab6d-211ab66b4ee4 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:57.933591 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-f4cf5e85-45db-49ae-9038-421724a91f64 admin admin] The token cache is being invalidate because project a50199051ad04d44aaa79918fa839beb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:39:57.933971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-f4cf5e85-45db-49ae-9038-421724a91f64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:57.934626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-f4cf5e85-45db-49ae-9038-421724a91f64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:57.936843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1251/2514] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:39:57 2022] DELETE /identity/v3/projects/a50199051ad04d44aaa79918fa839beb => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:39:57.941120 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf5e85-45db-49ae-9038-421724a91f64 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:57.942469 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-f4cf5e85-45db-49ae-9038-421724a91f64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:57.967609 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e26011f-9f81-4a20-889f-0b67380ae98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:57.968162 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26011f-9f81-4a20-889f-0b67380ae98b admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:57.968447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26011f-9f81-4a20-889f-0b67380ae98b admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:57.968713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26011f-9f81-4a20-889f-0b67380ae98b admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:57.969363 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e26011f-9f81-4a20-889f-0b67380ae98b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:57.971035 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e26011f-9f81-4a20-889f-0b67380ae98b admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:57.981109 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1264/2515] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:39:57.985775 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e26011f-9f81-4a20-889f-0b67380ae98b admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:57.987521 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-3e26011f-9f81-4a20-889f-0b67380ae98b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:58.000313 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e26011f-9f81-4a20-889f-0b67380ae98b admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:58.005080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eba2c6-6448-488a-b9c9-99710f9444e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:58.005722 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eba2c6-6448-488a-b9c9-99710f9444e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:58.005995 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eba2c6-6448-488a-b9c9-99710f9444e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:58.006283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eba2c6-6448-488a-b9c9-99710f9444e7 admin admin] PATH_INFO: `/v3/projects/26a23f5b960f4041b13acea131ec6a31/users/7fe8456c36af4f6c8d8aab5d5eb5a0ce/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:58.009835 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf5e85-45db-49ae-9038-421724a91f64 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:58.015468 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570efbd6-0f12-415c-b375-35532fd1e8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:58.016096 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570efbd6-0f12-415c-b375-35532fd1e8bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:58.016403 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570efbd6-0f12-415c-b375-35532fd1e8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:58.016678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570efbd6-0f12-415c-b375-35532fd1e8bb admin admin] PATH_INFO: `/v3/users/fac3cbc4319f4da2bc0baa93fc9c04a7` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:58.017837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570efbd6-0f12-415c-b375-35532fd1e8bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=fac3cbc4319f4da2bc0baa93fc9c04a7)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:58.019814 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570efbd6-0f12-415c-b375-35532fd1e8bb admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:58.024961 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eba2c6-6448-488a-b9c9-99710f9444e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26a23f5b960f4041b13acea131ec6a31, user_id=7fe8456c36af4f6c8d8aab5d5eb5a0ce, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:58.024961 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eba2c6-6448-488a-b9c9-99710f9444e7 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:58.122617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1265/2516] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:39:57 2022] PUT /identity/v3/projects/26a23f5b960f4041b13acea131ec6a31/users/7fe8456c36af4f6c8d8aab5d5eb5a0ce/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:39:58.126304 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eba2c6-6448-488a-b9c9-99710f9444e7 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:58.188193 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:58.192432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eba2c6-6448-488a-b9c9-99710f9444e7 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:58.197469 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149f590-e2c6-40f9-8e29-92d20a97c91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:58.198097 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149f590-e2c6-40f9-8e29-92d20a97c91b admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:58.198374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149f590-e2c6-40f9-8e29-92d20a97c91b admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:58.198671 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149f590-e2c6-40f9-8e29-92d20a97c91b admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:58.199318 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5149f590-e2c6-40f9-8e29-92d20a97c91b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:58.200916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5149f590-e2c6-40f9-8e29-92d20a97c91b admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:58.212263 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1266/2517] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:39:58 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:39:58.216610 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149f590-e2c6-40f9-8e29-92d20a97c91b admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:58.220831 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-570efbd6-0f12-415c-b375-35532fd1e8bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fac3cbc4319f4da2bc0baa93fc9c04a7'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:58.227220 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149f590-e2c6-40f9-8e29-92d20a97c91b admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:58.232008 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ccf7a7-69f6-4aa7-b8f6-7a6a21d47837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:58.242961 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ccf7a7-69f6-4aa7-b8f6-7a6a21d47837 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:58.242961 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ccf7a7-69f6-4aa7-b8f6-7a6a21d47837 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:58.242961 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ccf7a7-69f6-4aa7-b8f6-7a6a21d47837 admin admin] PATH_INFO: `/v3/projects/26a23f5b960f4041b13acea131ec6a31/users/7fe8456c36af4f6c8d8aab5d5eb5a0ce/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:58.242961 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ccf7a7-69f6-4aa7-b8f6-7a6a21d47837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26a23f5b960f4041b13acea131ec6a31, user_id=7fe8456c36af4f6c8d8aab5d5eb5a0ce, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:58.242961 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ccf7a7-69f6-4aa7-b8f6-7a6a21d47837 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:58.249809 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1267/2518] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:39:58 2022] PUT /identity/v3/projects/26a23f5b960f4041b13acea131ec6a31/users/7fe8456c36af4f6c8d8aab5d5eb5a0ce/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:39:58.261491 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b1ae6-48c7-4f70-8c69-0d684edbff2c None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:58.261677 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b1ae6-48c7-4f70-8c69-0d684edbff2c None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:58.261953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b1ae6-48c7-4f70-8c69-0d684edbff2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:58.291964 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-f89b1ae6-48c7-4f70-8c69-0d684edbff2c None None] MFA Rules not processed for user `7fe8456c36af4f6c8d8aab5d5eb5a0ce`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:39:58.294072 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-f89b1ae6-48c7-4f70-8c69-0d684edbff2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:58.344706 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-570efbd6-0f12-415c-b375-35532fd1e8bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fac3cbc4319f4da2bc0baa93fc9c04a7'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:58.345505 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-570efbd6-0f12-415c-b375-35532fd1e8bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fac3cbc4319f4da2bc0baa93fc9c04a7'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:58.345989 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-570efbd6-0f12-415c-b375-35532fd1e8bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fac3cbc4319f4da2bc0baa93fc9c04a7'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:58.357193 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:58.359933 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-570efbd6-0f12-415c-b375-35532fd1e8bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fac3cbc4319f4da2bc0baa93fc9c04a7'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:58.367189 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1268/2519] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:39:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:39:58.376523 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1252/2520] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:39:57 2022] DELETE /identity/v3/users/fac3cbc4319f4da2bc0baa93fc9c04a7 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:39:58.381026 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b1ae6-48c7-4f70-8c69-0d684edbff2c None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:58.385215 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-f89b1ae6-48c7-4f70-8c69-0d684edbff2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:58.447605 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b1ae6-48c7-4f70-8c69-0d684edbff2c None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:58.506325 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc82689-5efa-45a4-8aea-83b3aabf3e6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:58.506968 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc82689-5efa-45a4-8aea-83b3aabf3e6e service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:58.507247 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc82689-5efa-45a4-8aea-83b3aabf3e6e service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:58.507519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc82689-5efa-45a4-8aea-83b3aabf3e6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:58.542685 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:58.573972 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc82689-5efa-45a4-8aea-83b3aabf3e6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:58.575469 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc82689-5efa-45a4-8aea-83b3aabf3e6e service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:58.586798 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1269/2521] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:39:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:39:58.653177 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570efbd6-0f12-415c-b375-35532fd1e8bb admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:58.656815 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-570efbd6-0f12-415c-b375-35532fd1e8bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:58.783842 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570efbd6-0f12-415c-b375-35532fd1e8bb admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:58.789953 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc8f0e1-e9e5-4bd0-a0b1-48e488a8bf35 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3db46a4b4e74cc0bb2a7bd6f0e99105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:58.790609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8f0e1-e9e5-4bd0-a0b1-48e488a8bf35 service placement] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:58.790891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8f0e1-e9e5-4bd0-a0b1-48e488a8bf35 service placement] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:58.791164 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8f0e1-e9e5-4bd0-a0b1-48e488a8bf35 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:58.793018 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-1bc8f0e1-e9e5-4bd0-a0b1-48e488a8bf35 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:58.806241 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc8f0e1-e9e5-4bd0-a0b1-48e488a8bf35 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:58.807875 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc8f0e1-e9e5-4bd0-a0b1-48e488a8bf35 service placement] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:58.809872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc82689-5efa-45a4-8aea-83b3aabf3e6e service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:58.811866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-ecc82689-5efa-45a4-8aea-83b3aabf3e6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:39:58.819321 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1253/2522] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:39:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:39:58.921248 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc82689-5efa-45a4-8aea-83b3aabf3e6e service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:58.925750 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99075ed4-52fd-445d-9adf-959091f1b4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:58.926374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99075ed4-52fd-445d-9adf-959091f1b4e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:58.926674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99075ed4-52fd-445d-9adf-959091f1b4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:58.926949 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99075ed4-52fd-445d-9adf-959091f1b4e1 admin admin] PATH_INFO: `/v3/projects/3597726dc4a7402ba189f4ba26aeaed3` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:58.928060 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99075ed4-52fd-445d-9adf-959091f1b4e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3597726dc4a7402ba189f4ba26aeaed3)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:58.929959 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99075ed4-52fd-445d-9adf-959091f1b4e1 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:59.089168 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-99075ed4-52fd-445d-9adf-959091f1b4e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3597726dc4a7402ba189f4ba26aeaed3', 'request_id': 'req-99075ed4-52fd-445d-9adf-959091f1b4e1'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:59.185352 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-99075ed4-52fd-445d-9adf-959091f1b4e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3597726dc4a7402ba189f4ba26aeaed3', 'request_id': 'req-99075ed4-52fd-445d-9adf-959091f1b4e1'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:59.192648 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-99075ed4-52fd-445d-9adf-959091f1b4e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3597726dc4a7402ba189f4ba26aeaed3', 'request_id': 'req-99075ed4-52fd-445d-9adf-959091f1b4e1'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:59.207868 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-99075ed4-52fd-445d-9adf-959091f1b4e1 admin admin] The token cache is being invalidate because project 3597726dc4a7402ba189f4ba26aeaed3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:39:59.208477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-99075ed4-52fd-445d-9adf-959091f1b4e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:59.209295 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-99075ed4-52fd-445d-9adf-959091f1b4e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:59.211011 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1270/2523] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:39:58 2022] DELETE /identity/v3/projects/3597726dc4a7402ba189f4ba26aeaed3 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:59.215169 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc8f0e1-e9e5-4bd0-a0b1-48e488a8bf35 service placement] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:59.227178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc8f0e1-e9e5-4bd0-a0b1-48e488a8bf35 service placement] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:39:59.289582 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde1818-2b61-428e-a3d1-9b031b98d173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:39:59.290198 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde1818-2b61-428e-a3d1-9b031b98d173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:39:59.290501 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde1818-2b61-428e-a3d1-9b031b98d173 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:39:59.290773 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde1818-2b61-428e-a3d1-9b031b98d173 admin admin] PATH_INFO: `/v3/users/2843cd5feab8472bbbc9ad0930be7176` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:39:59.291953 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde1818-2b61-428e-a3d1-9b031b98d173 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2843cd5feab8472bbbc9ad0930be7176)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:39:59.293847 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde1818-2b61-428e-a3d1-9b031b98d173 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:39:59.564210 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-0cde1818-2b61-428e-a3d1-9b031b98d173 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2843cd5feab8472bbbc9ad0930be7176'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:59.653084 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-0cde1818-2b61-428e-a3d1-9b031b98d173 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2843cd5feab8472bbbc9ad0930be7176'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:59.653641 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-0cde1818-2b61-428e-a3d1-9b031b98d173 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2843cd5feab8472bbbc9ad0930be7176'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:59.655009 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-0cde1818-2b61-428e-a3d1-9b031b98d173 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2843cd5feab8472bbbc9ad0930be7176'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:59.668250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-0cde1818-2b61-428e-a3d1-9b031b98d173 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2843cd5feab8472bbbc9ad0930be7176'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:39:59.673334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:39:59.682228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1254/2524] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:39:59 2022] DELETE /identity/v3/users/2843cd5feab8472bbbc9ad0930be7176 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:39:59.975542 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99075ed4-52fd-445d-9adf-959091f1b4e1 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:39:59.977820 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-99075ed4-52fd-445d-9adf-959091f1b4e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:00.050908 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99075ed4-52fd-445d-9adf-959091f1b4e1 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:00.064846 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:00.102736 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8289b36a-b41d-4507-ba08-af3704140dbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:00.103348 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289b36a-b41d-4507-ba08-af3704140dbe service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:00.103629 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289b36a-b41d-4507-ba08-af3704140dbe service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:00.103900 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289b36a-b41d-4507-ba08-af3704140dbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:00.105659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-8289b36a-b41d-4507-ba08-af3704140dbe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:00.145533 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde1818-2b61-428e-a3d1-9b031b98d173 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:00.147535 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-0cde1818-2b61-428e-a3d1-9b031b98d173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:00.189640 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8289b36a-b41d-4507-ba08-af3704140dbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:00.191301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8289b36a-b41d-4507-ba08-af3704140dbe service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:00.202372 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1271/2525] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:39:59 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:40:00.233094 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde1818-2b61-428e-a3d1-9b031b98d173 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:00.244164 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a7ac9-3319-4aea-bc7f-a4d58673bb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:00.244164 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a7ac9-3319-4aea-bc7f-a4d58673bb1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:00.244164 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a7ac9-3319-4aea-bc7f-a4d58673bb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:00.244164 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a7ac9-3319-4aea-bc7f-a4d58673bb1a admin admin] PATH_INFO: `/v3/projects/750d9c8d7db140deab48a6580625ffa7` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:00.244426 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029a7ac9-3319-4aea-bc7f-a4d58673bb1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=750d9c8d7db140deab48a6580625ffa7)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:00.244426 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029a7ac9-3319-4aea-bc7f-a4d58673bb1a admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:00.446950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-029a7ac9-3319-4aea-bc7f-a4d58673bb1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '750d9c8d7db140deab48a6580625ffa7', 'request_id': 'req-029a7ac9-3319-4aea-bc7f-a4d58673bb1a'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:00.453572 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-029a7ac9-3319-4aea-bc7f-a4d58673bb1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '750d9c8d7db140deab48a6580625ffa7', 'request_id': 'req-029a7ac9-3319-4aea-bc7f-a4d58673bb1a'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:00.460533 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-029a7ac9-3319-4aea-bc7f-a4d58673bb1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '750d9c8d7db140deab48a6580625ffa7', 'request_id': 'req-029a7ac9-3319-4aea-bc7f-a4d58673bb1a'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:00.590839 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-029a7ac9-3319-4aea-bc7f-a4d58673bb1a admin admin] The token cache is being invalidate because project 750d9c8d7db140deab48a6580625ffa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:40:00.591230 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-029a7ac9-3319-4aea-bc7f-a4d58673bb1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:00.592414 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-029a7ac9-3319-4aea-bc7f-a4d58673bb1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:00.594622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1255/2526] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:40:00 2022] DELETE /identity/v3/projects/750d9c8d7db140deab48a6580625ffa7 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:00.599612 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a67045-7b74-49f1-b1a6-091d947e5ed1 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:00.599948 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a67045-7b74-49f1-b1a6-091d947e5ed1 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:00.600240 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a67045-7b74-49f1-b1a6-091d947e5ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:00.622927 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-80a67045-7b74-49f1-b1a6-091d947e5ed1 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:00.624661 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-80a67045-7b74-49f1-b1a6-091d947e5ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:00.697685 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1272/2527] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:40:00.706745 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d631ee9-d412-4043-87f1-258668b53729 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:00.707304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d631ee9-d412-4043-87f1-258668b53729 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:00.707304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d631ee9-d412-4043-87f1-258668b53729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:00.731193 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-0d631ee9-d412-4043-87f1-258668b53729 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:00.733071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-0d631ee9-d412-4043-87f1-258668b53729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:00.741969 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1256/2528] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:40:00.746729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a67045-7b74-49f1-b1a6-091d947e5ed1 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:00.757607 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a67045-7b74-49f1-b1a6-091d947e5ed1 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:00.762365 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ff057-629f-47ed-b249-03f7dc033fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:00.766004 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ff057-629f-47ed-b249-03f7dc033fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:00.766004 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ff057-629f-47ed-b249-03f7dc033fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:00.766004 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ff057-629f-47ed-b249-03f7dc033fb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:00.766004 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ff057-629f-47ed-b249-03f7dc033fb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:40:00.766004 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ff057-629f-47ed-b249-03f7dc033fb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:00.766747 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ff057-629f-47ed-b249-03f7dc033fb1 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:00.779089 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1273/2529] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:40:00.786532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d631ee9-d412-4043-87f1-258668b53729 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:00.797370 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d631ee9-d412-4043-87f1-258668b53729 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:00.802358 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494a426-c094-4de6-bf38-1651d91e755c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:00.802988 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494a426-c094-4de6-bf38-1651d91e755c admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:00.803258 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494a426-c094-4de6-bf38-1651d91e755c admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:00.803557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494a426-c094-4de6-bf38-1651d91e755c admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:00.804189 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5494a426-c094-4de6-bf38-1651d91e755c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:00.806137 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5494a426-c094-4de6-bf38-1651d91e755c admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:00.903222 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1257/2530] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:40:00 2022] POST /identity/v3/projects => generated 369 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:40:00.906499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ff057-629f-47ed-b249-03f7dc033fb1 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:00.924559 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ff057-629f-47ed-b249-03f7dc033fb1 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:00.998305 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ce735-7af8-4f3b-9758-200f0ced5883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:00.999277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ce735-7af8-4f3b-9758-200f0ced5883 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:00.999632 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ce735-7af8-4f3b-9758-200f0ced5883 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:01.000101 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ce735-7af8-4f3b-9758-200f0ced5883 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:01.000743 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ce735-7af8-4f3b-9758-200f0ced5883 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:01.002834 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ce735-7af8-4f3b-9758-200f0ced5883 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:01.012005 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=100 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:01.145838 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1274/2531] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:40:00 2022] POST /identity/v3/users => generated 285 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:40:01.150307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494a426-c094-4de6-bf38-1651d91e755c admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:01.160761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494a426-c094-4de6-bf38-1651d91e755c admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:01.165860 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086a95c-368a-4dd4-a02c-f6555a4e4556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:01.166540 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086a95c-368a-4dd4-a02c-f6555a4e4556 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:01.166819 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086a95c-368a-4dd4-a02c-f6555a4e4556 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:01.167089 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086a95c-368a-4dd4-a02c-f6555a4e4556 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:01.167721 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c086a95c-368a-4dd4-a02c-f6555a4e4556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:01.169331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c086a95c-368a-4dd4-a02c-f6555a4e4556 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:01.179943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1258/2532] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:40:01.184057 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ce735-7af8-4f3b-9758-200f0ced5883 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:01.195051 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ce735-7af8-4f3b-9758-200f0ced5883 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:01.199924 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633afb4-57fd-426f-9826-b65dbc5f8fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:01.206032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633afb4-57fd-426f-9826-b65dbc5f8fba admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:01.206032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633afb4-57fd-426f-9826-b65dbc5f8fba admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:01.206032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633afb4-57fd-426f-9826-b65dbc5f8fba admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:01.206032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b633afb4-57fd-426f-9826-b65dbc5f8fba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:01.206032 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b633afb4-57fd-426f-9826-b65dbc5f8fba admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:01.215067 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1275/2533] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:40:01.219302 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086a95c-368a-4dd4-a02c-f6555a4e4556 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:01.229309 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086a95c-368a-4dd4-a02c-f6555a4e4556 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:01.233880 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ccc70-1588-4be2-b206-2b8e728c483e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:01.234566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ccc70-1588-4be2-b206-2b8e728c483e admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:01.235667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ccc70-1588-4be2-b206-2b8e728c483e admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:01.235667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ccc70-1588-4be2-b206-2b8e728c483e admin admin] PATH_INFO: `/v3/projects/ae0704b453a64640901eea2b74ac5c8a/users/b0527e7febd040c897ab9321598638a9/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:01.252137 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78ccc70-1588-4be2-b206-2b8e728c483e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae0704b453a64640901eea2b74ac5c8a, user_id=b0527e7febd040c897ab9321598638a9, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:01.254819 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78ccc70-1588-4be2-b206-2b8e728c483e admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:01.362484 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1259/2534] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:40:01 2022] PUT /identity/v3/projects/ae0704b453a64640901eea2b74ac5c8a/users/b0527e7febd040c897ab9321598638a9/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:01.373813 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654536f-3592-4389-bd20-f5b8f5ba36df None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:01.374069 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654536f-3592-4389-bd20-f5b8f5ba36df None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:01.374370 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654536f-3592-4389-bd20-f5b8f5ba36df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:01.405763 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-f654536f-3592-4389-bd20-f5b8f5ba36df None None] MFA Rules not processed for user `b0527e7febd040c897ab9321598638a9`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:01.407291 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-f654536f-3592-4389-bd20-f5b8f5ba36df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:01.497268 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1276/2535] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:40:01.508997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ccc70-1588-4be2-b206-2b8e728c483e admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:01.510717 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-c78ccc70-1588-4be2-b206-2b8e728c483e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:01.535743 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:01.590643 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ccc70-1588-4be2-b206-2b8e728c483e admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:01.595569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8050a1e5-b2a4-4555-aedd-bb6887273b80 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2e72c8e7b444e978bf99c6404febcd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:01.596177 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050a1e5-b2a4-4555-aedd-bb6887273b80 service glance] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:01.596448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050a1e5-b2a4-4555-aedd-bb6887273b80 service glance] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:01.596721 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050a1e5-b2a4-4555-aedd-bb6887273b80 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:01.605341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8050a1e5-b2a4-4555-aedd-bb6887273b80 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:01.606795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8050a1e5-b2a4-4555-aedd-bb6887273b80 service glance] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:01.616017 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1260/2536] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:40:01 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:40:05.288733 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654536f-3592-4389-bd20-f5b8f5ba36df None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:05.290278 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-f654536f-3592-4389-bd20-f5b8f5ba36df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:05.378574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654536f-3592-4389-bd20-f5b8f5ba36df None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:05.383012 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a3633-e4d3-4133-a11a-753a92e7392a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:05.383658 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a3633-e4d3-4133-a11a-753a92e7392a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:05.383935 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a3633-e4d3-4133-a11a-753a92e7392a admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:05.384208 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a3633-e4d3-4133-a11a-753a92e7392a admin admin] PATH_INFO: `/v3/users/82856c346db54659aa29107daadceafe` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:05.385351 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a3633-e4d3-4133-a11a-753a92e7392a admin admin] RBAC: Authorizing `identity:delete_user(user_id=82856c346db54659aa29107daadceafe)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:05.387339 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a3633-e4d3-4133-a11a-753a92e7392a admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:05.569286 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-9f6a3633-e4d3-4133-a11a-753a92e7392a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82856c346db54659aa29107daadceafe'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:05.581618 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-9f6a3633-e4d3-4133-a11a-753a92e7392a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82856c346db54659aa29107daadceafe'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:05.586366 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:05.589283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-9f6a3633-e4d3-4133-a11a-753a92e7392a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82856c346db54659aa29107daadceafe'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:05.676100 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-9f6a3633-e4d3-4133-a11a-753a92e7392a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82856c346db54659aa29107daadceafe'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:05.676949 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-9f6a3633-e4d3-4133-a11a-753a92e7392a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82856c346db54659aa29107daadceafe'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:05.684093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1277/2537] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:40:05 2022] DELETE /identity/v3/users/82856c346db54659aa29107daadceafe => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:05.688793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704f60c-efea-4ff9-ae26-8854b2c3de6c None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:05.689350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704f60c-efea-4ff9-ae26-8854b2c3de6c None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:05.689476 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704f60c-efea-4ff9-ae26-8854b2c3de6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:05.711065 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-c704f60c-efea-4ff9-ae26-8854b2c3de6c None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:05.712362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-c704f60c-efea-4ff9-ae26-8854b2c3de6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:05.779808 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1261/2538] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:40:05.789655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a3633-e4d3-4133-a11a-753a92e7392a admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:05.791278 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-9f6a3633-e4d3-4133-a11a-753a92e7392a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:05.854207 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a3633-e4d3-4133-a11a-753a92e7392a admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:05.858919 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52191bc9-cac0-4149-bf6e-1ad32e82436e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:05.859530 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52191bc9-cac0-4149-bf6e-1ad32e82436e service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:05.859805 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52191bc9-cac0-4149-bf6e-1ad32e82436e service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:05.860072 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52191bc9-cac0-4149-bf6e-1ad32e82436e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:05.869590 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52191bc9-cac0-4149-bf6e-1ad32e82436e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:05.871089 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52191bc9-cac0-4149-bf6e-1ad32e82436e service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:05.883418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1278/2539] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:40:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:40:06.572892 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704f60c-efea-4ff9-ae26-8854b2c3de6c None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:06.575279 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-c704f60c-efea-4ff9-ae26-8854b2c3de6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:06.601432 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704f60c-efea-4ff9-ae26-8854b2c3de6c None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:06.612322 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f318b7e5-72fb-4d05-a4ef-2abfc0125d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:06.612994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318b7e5-72fb-4d05-a4ef-2abfc0125d72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:06.613300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318b7e5-72fb-4d05-a4ef-2abfc0125d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:06.613572 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318b7e5-72fb-4d05-a4ef-2abfc0125d72 admin admin] PATH_INFO: `/v3/projects/25ca49c56fdc409b993ec34f5d794044` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:06.614804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f318b7e5-72fb-4d05-a4ef-2abfc0125d72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25ca49c56fdc409b993ec34f5d794044)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:06.616742 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f318b7e5-72fb-4d05-a4ef-2abfc0125d72 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:06.669295 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52191bc9-cac0-4149-bf6e-1ad32e82436e service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:06.671463 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-52191bc9-cac0-4149-bf6e-1ad32e82436e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:06.727433 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-f318b7e5-72fb-4d05-a4ef-2abfc0125d72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25ca49c56fdc409b993ec34f5d794044', 'request_id': 'req-f318b7e5-72fb-4d05-a4ef-2abfc0125d72'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:06.742947 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-f318b7e5-72fb-4d05-a4ef-2abfc0125d72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25ca49c56fdc409b993ec34f5d794044', 'request_id': 'req-f318b7e5-72fb-4d05-a4ef-2abfc0125d72'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:06.750000 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52191bc9-cac0-4149-bf6e-1ad32e82436e service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:06.756171 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-f318b7e5-72fb-4d05-a4ef-2abfc0125d72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25ca49c56fdc409b993ec34f5d794044', 'request_id': 'req-f318b7e5-72fb-4d05-a4ef-2abfc0125d72'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:06.827184 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568d128-7372-444e-a6fc-c6e43a70743b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:06.827986 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568d128-7372-444e-a6fc-c6e43a70743b service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:06.828420 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568d128-7372-444e-a6fc-c6e43a70743b service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:06.828702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568d128-7372-444e-a6fc-c6e43a70743b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:06.831106 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-c568d128-7372-444e-a6fc-c6e43a70743b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:06.838806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-f318b7e5-72fb-4d05-a4ef-2abfc0125d72 admin admin] The token cache is being invalidate because project 25ca49c56fdc409b993ec34f5d794044 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:40:06.839379 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-f318b7e5-72fb-4d05-a4ef-2abfc0125d72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:06.840377 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-f318b7e5-72fb-4d05-a4ef-2abfc0125d72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:06.846814 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1262/2540] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:40:06 2022] DELETE /identity/v3/projects/25ca49c56fdc409b993ec34f5d794044 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:06.850531 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f318b7e5-72fb-4d05-a4ef-2abfc0125d72 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:06.852277 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-f318b7e5-72fb-4d05-a4ef-2abfc0125d72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:06.887753 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:06.906255 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568d128-7372-444e-a6fc-c6e43a70743b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:06.907821 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568d128-7372-444e-a6fc-c6e43a70743b service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:06.912954 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:06.917969 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1279/2541] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:40:06 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:40:06.926687 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f318b7e5-72fb-4d05-a4ef-2abfc0125d72 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:06.931790 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf30fda-4d35-41bf-9150-742e8f6ee8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:06.932421 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf30fda-4d35-41bf-9150-742e8f6ee8d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:06.932698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf30fda-4d35-41bf-9150-742e8f6ee8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:06.932974 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf30fda-4d35-41bf-9150-742e8f6ee8d1 admin admin] PATH_INFO: `/v3/users/8aba47a3a35b41419560e25cf3e4ae42` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:06.934134 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf30fda-4d35-41bf-9150-742e8f6ee8d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8aba47a3a35b41419560e25cf3e4ae42)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:06.936576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf30fda-4d35-41bf-9150-742e8f6ee8d1 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:07.169332 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5bf30fda-4d35-41bf-9150-742e8f6ee8d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8aba47a3a35b41419560e25cf3e4ae42'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:07.333515 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5bf30fda-4d35-41bf-9150-742e8f6ee8d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8aba47a3a35b41419560e25cf3e4ae42'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:07.333988 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5bf30fda-4d35-41bf-9150-742e8f6ee8d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8aba47a3a35b41419560e25cf3e4ae42'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:07.334618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5bf30fda-4d35-41bf-9150-742e8f6ee8d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8aba47a3a35b41419560e25cf3e4ae42'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:07.347828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5bf30fda-4d35-41bf-9150-742e8f6ee8d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8aba47a3a35b41419560e25cf3e4ae42'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:07.363372 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1263/2542] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:40:06 2022] DELETE /identity/v3/users/8aba47a3a35b41419560e25cf3e4ae42 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:07.821231 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568d128-7372-444e-a6fc-c6e43a70743b service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:07.835978 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568d128-7372-444e-a6fc-c6e43a70743b service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:07.921423 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a4e0d5-746a-4463-8231-0d5ccc77ba6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:07.922059 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4e0d5-746a-4463-8231-0d5ccc77ba6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:07.922332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4e0d5-746a-4463-8231-0d5ccc77ba6c admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:07.922653 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4e0d5-746a-4463-8231-0d5ccc77ba6c admin admin] PATH_INFO: `/v3/projects/5ceaea556ace4ac4aec6e9fe964b2e12` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:07.923792 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a4e0d5-746a-4463-8231-0d5ccc77ba6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ceaea556ace4ac4aec6e9fe964b2e12)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:07.925816 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a4e0d5-746a-4463-8231-0d5ccc77ba6c admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:08.070261 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c3a4e0d5-746a-4463-8231-0d5ccc77ba6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ceaea556ace4ac4aec6e9fe964b2e12', 'request_id': 'req-c3a4e0d5-746a-4463-8231-0d5ccc77ba6c'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:08.172838 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c3a4e0d5-746a-4463-8231-0d5ccc77ba6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ceaea556ace4ac4aec6e9fe964b2e12', 'request_id': 'req-c3a4e0d5-746a-4463-8231-0d5ccc77ba6c'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:08.179143 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c3a4e0d5-746a-4463-8231-0d5ccc77ba6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ceaea556ace4ac4aec6e9fe964b2e12', 'request_id': 'req-c3a4e0d5-746a-4463-8231-0d5ccc77ba6c'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:08.191246 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c3a4e0d5-746a-4463-8231-0d5ccc77ba6c admin admin] The token cache is being invalidate because project 5ceaea556ace4ac4aec6e9fe964b2e12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:40:08.191541 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c3a4e0d5-746a-4463-8231-0d5ccc77ba6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:08.192176 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c3a4e0d5-746a-4463-8231-0d5ccc77ba6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:08.194625 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1280/2543] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:40:07 2022] DELETE /identity/v3/projects/5ceaea556ace4ac4aec6e9fe964b2e12 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:08.198846 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c4349-2d94-4f95-b516-0a28ddfb9777 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:08.199295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c4349-2d94-4f95-b516-0a28ddfb9777 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:08.199393 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c4349-2d94-4f95-b516-0a28ddfb9777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:08.220556 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-fe3c4349-2d94-4f95-b516-0a28ddfb9777 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:08.222598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-fe3c4349-2d94-4f95-b516-0a28ddfb9777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:08.288780 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1264/2544] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:40:08.296952 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b22387-a450-41a2-a480-d0ae05d19ee9 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:08.297235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b22387-a450-41a2-a480-d0ae05d19ee9 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:08.297508 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b22387-a450-41a2-a480-d0ae05d19ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:08.316996 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-f8b22387-a450-41a2-a480-d0ae05d19ee9 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:08.318192 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-f8b22387-a450-41a2-a480-d0ae05d19ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:08.326858 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1281/2545] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:40:08.331079 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c4349-2d94-4f95-b516-0a28ddfb9777 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:08.341311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c4349-2d94-4f95-b516-0a28ddfb9777 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:08.346994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c519ba-d79b-42a0-bfc6-a082f699e307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:08.347513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c519ba-d79b-42a0-bfc6-a082f699e307 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:08.347820 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c519ba-d79b-42a0-bfc6-a082f699e307 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:08.348088 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c519ba-d79b-42a0-bfc6-a082f699e307 admin admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:08.348740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c519ba-d79b-42a0-bfc6-a082f699e307 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:40:08.349040 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c519ba-d79b-42a0-bfc6-a082f699e307 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:08.350703 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c519ba-d79b-42a0-bfc6-a082f699e307 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:08.362887 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1265/2546] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:40:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:40:08.367586 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b22387-a450-41a2-a480-d0ae05d19ee9 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:08.377277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b22387-a450-41a2-a480-d0ae05d19ee9 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:08.382031 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bae27-017a-4d96-bc78-13e6879bd4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:08.382686 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bae27-017a-4d96-bc78-13e6879bd4ec admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:08.383113 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bae27-017a-4d96-bc78-13e6879bd4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:08.383387 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bae27-017a-4d96-bc78-13e6879bd4ec admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:08.384031 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888bae27-017a-4d96-bc78-13e6879bd4ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:08.385926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888bae27-017a-4d96-bc78-13e6879bd4ec admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:08.490013 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1282/2547] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:40:08 2022] POST /identity/v3/projects => generated 407 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:40:08.494206 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c519ba-d79b-42a0-bfc6-a082f699e307 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:08.504489 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c519ba-d79b-42a0-bfc6-a082f699e307 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:08.534550 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:08.571626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa176df-4070-46c6-8f23-2710283bd8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:08.572563 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa176df-4070-46c6-8f23-2710283bd8e1 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:08.573084 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa176df-4070-46c6-8f23-2710283bd8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:08.574087 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa176df-4070-46c6-8f23-2710283bd8e1 admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:08.574804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa176df-4070-46c6-8f23-2710283bd8e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:08.577913 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa176df-4070-46c6-8f23-2710283bd8e1 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:08.688599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1266/2548] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:40:08 2022] POST /identity/v3/users => generated 304 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:40:08.695118 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bae27-017a-4d96-bc78-13e6879bd4ec admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:08.707025 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bae27-017a-4d96-bc78-13e6879bd4ec admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:08.712285 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c9c875-af80-42dd-bc03-30d1065bfc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:08.714313 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9c875-af80-42dd-bc03-30d1065bfc29 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:08.714313 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9c875-af80-42dd-bc03-30d1065bfc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:08.714313 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9c875-af80-42dd-bc03-30d1065bfc29 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:08.714313 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c9c875-af80-42dd-bc03-30d1065bfc29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:08.715760 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c9c875-af80-42dd-bc03-30d1065bfc29 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:08.726025 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1283/2549] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:40:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:40:08.730553 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa176df-4070-46c6-8f23-2710283bd8e1 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:08.744082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa176df-4070-46c6-8f23-2710283bd8e1 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:08.750557 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9dbbd-c134-4a14-94fa-7b7df15bf959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:08.751441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9dbbd-c134-4a14-94fa-7b7df15bf959 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:08.751833 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9dbbd-c134-4a14-94fa-7b7df15bf959 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:08.752103 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9dbbd-c134-4a14-94fa-7b7df15bf959 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:08.752752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9dbbd-c134-4a14-94fa-7b7df15bf959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:08.754361 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9dbbd-c134-4a14-94fa-7b7df15bf959 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:08.768503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1267/2550] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:40:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:40:08.779166 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c9c875-af80-42dd-bc03-30d1065bfc29 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:08.790016 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c9c875-af80-42dd-bc03-30d1065bfc29 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:08.795186 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a02f46-7d17-430c-982c-66530eccaf82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:08.796133 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a02f46-7d17-430c-982c-66530eccaf82 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:08.796480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a02f46-7d17-430c-982c-66530eccaf82 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:08.796822 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a02f46-7d17-430c-982c-66530eccaf82 admin admin] PATH_INFO: `/v3/projects/a1dcff769e8d44909c60cbdb920d253e/users/28f9e182f32b48bd84bdd94f28884f4c/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:08.812571 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a02f46-7d17-430c-982c-66530eccaf82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1dcff769e8d44909c60cbdb920d253e, user_id=28f9e182f32b48bd84bdd94f28884f4c, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:08.815264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a02f46-7d17-430c-982c-66530eccaf82 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:08.905354 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1284/2551] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:40:08 2022] PUT /identity/v3/projects/a1dcff769e8d44909c60cbdb920d253e/users/28f9e182f32b48bd84bdd94f28884f4c/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:08.977105 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da9170-0dda-4169-861a-8acd942be395 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:08.977424 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da9170-0dda-4169-861a-8acd942be395 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:08.980746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da9170-0dda-4169-861a-8acd942be395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:09.013839 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-e5da9170-0dda-4169-861a-8acd942be395 None None] MFA Rules not processed for user `28f9e182f32b48bd84bdd94f28884f4c`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:09.015196 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-e5da9170-0dda-4169-861a-8acd942be395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:09.087940 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1268/2552] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:08 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:40:09.093225 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a02f46-7d17-430c-982c-66530eccaf82 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:09.136967 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:09.171580 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a02f46-7d17-430c-982c-66530eccaf82 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:09.176340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863544e-de27-4585-9844-e12e1ce964d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:09.176933 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863544e-de27-4585-9844-e12e1ce964d5 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:09.177210 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863544e-de27-4585-9844-e12e1ce964d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:09.177481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863544e-de27-4585-9844-e12e1ce964d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:09.178102 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863544e-de27-4585-9844-e12e1ce964d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:09.180030 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863544e-de27-4585-9844-e12e1ce964d5 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:09.312694 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1285/2553] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:40:09 2022] POST /identity/v3/projects => generated 407 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:40:09.316712 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da9170-0dda-4169-861a-8acd942be395 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:09.327213 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da9170-0dda-4169-861a-8acd942be395 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:09.383817 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54fc88-d0bc-45b0-9527-b5a4a2c112ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:09.384520 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54fc88-d0bc-45b0-9527-b5a4a2c112ef admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:09.384702 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54fc88-d0bc-45b0-9527-b5a4a2c112ef admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:09.384972 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54fc88-d0bc-45b0-9527-b5a4a2c112ef admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:09.385621 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb54fc88-d0bc-45b0-9527-b5a4a2c112ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:09.387567 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb54fc88-d0bc-45b0-9527-b5a4a2c112ef admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:09.510400 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1269/2554] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:40:09 2022] POST /identity/v3/users => generated 310 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:40:09.514633 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863544e-de27-4585-9844-e12e1ce964d5 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:09.524924 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863544e-de27-4585-9844-e12e1ce964d5 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:09.530066 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27229ce-5ccd-477b-a1f8-cfce4cc5d51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:09.530683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27229ce-5ccd-477b-a1f8-cfce4cc5d51e admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:09.530965 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27229ce-5ccd-477b-a1f8-cfce4cc5d51e admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:09.531265 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27229ce-5ccd-477b-a1f8-cfce4cc5d51e admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:09.531903 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27229ce-5ccd-477b-a1f8-cfce4cc5d51e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:09.533556 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27229ce-5ccd-477b-a1f8-cfce4cc5d51e admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:09.544523 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1286/2555] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:40:09.549671 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54fc88-d0bc-45b0-9527-b5a4a2c112ef admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:09.559885 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54fc88-d0bc-45b0-9527-b5a4a2c112ef admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:09.564744 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fc5d5-aa91-49d2-a39e-9be416608a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:09.565332 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fc5d5-aa91-49d2-a39e-9be416608a46 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:09.565638 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fc5d5-aa91-49d2-a39e-9be416608a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:09.565906 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fc5d5-aa91-49d2-a39e-9be416608a46 admin admin] PATH_INFO: `/v3/projects/471a719670534b67942fdec710b1a6a9/users/ff7c35457c3e43d1b912132785016789/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:09.580719 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1fc5d5-aa91-49d2-a39e-9be416608a46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=471a719670534b67942fdec710b1a6a9, user_id=ff7c35457c3e43d1b912132785016789, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:09.583701 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1fc5d5-aa91-49d2-a39e-9be416608a46 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:09.665044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1270/2556] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:40:09 2022] PUT /identity/v3/projects/471a719670534b67942fdec710b1a6a9/users/ff7c35457c3e43d1b912132785016789/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:09.669259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27229ce-5ccd-477b-a1f8-cfce4cc5d51e admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:09.680062 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27229ce-5ccd-477b-a1f8-cfce4cc5d51e admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:09.736796 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273117e2-b210-48bb-9c86-ca2ad50de47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:09.737478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273117e2-b210-48bb-9c86-ca2ad50de47e admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:09.737776 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273117e2-b210-48bb-9c86-ca2ad50de47e admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:09.738054 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273117e2-b210-48bb-9c86-ca2ad50de47e admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:09.738890 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273117e2-b210-48bb-9c86-ca2ad50de47e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:09.740527 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273117e2-b210-48bb-9c86-ca2ad50de47e admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:09.750964 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1287/2557] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:40:09.755262 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fc5d5-aa91-49d2-a39e-9be416608a46 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:09.767410 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fc5d5-aa91-49d2-a39e-9be416608a46 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:09.772130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa77568-cab5-4092-8abd-6063add76f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:09.772793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa77568-cab5-4092-8abd-6063add76f40 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:09.773066 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa77568-cab5-4092-8abd-6063add76f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:09.773342 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa77568-cab5-4092-8abd-6063add76f40 admin admin] PATH_INFO: `/v3/projects/471a719670534b67942fdec710b1a6a9/users/ff7c35457c3e43d1b912132785016789/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:09.775330 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa77568-cab5-4092-8abd-6063add76f40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=471a719670534b67942fdec710b1a6a9, user_id=ff7c35457c3e43d1b912132785016789, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:09.778017 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa77568-cab5-4092-8abd-6063add76f40 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:09.781023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:09.789625 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1271/2558] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:40:09 2022] PUT /identity/v3/projects/471a719670534b67942fdec710b1a6a9/users/ff7c35457c3e43d1b912132785016789/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:09.799682 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff048f0-a918-49d5-9f32-19f9033a8f59 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:09.800017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff048f0-a918-49d5-9f32-19f9033a8f59 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:09.800289 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff048f0-a918-49d5-9f32-19f9033a8f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:09.834490 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-bff048f0-a918-49d5-9f32-19f9033a8f59 None None] MFA Rules not processed for user `ff7c35457c3e43d1b912132785016789`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:09.835779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-bff048f0-a918-49d5-9f32-19f9033a8f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:09.907294 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1288/2559] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:09 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:40:09.917963 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa77568-cab5-4092-8abd-6063add76f40 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:09.920210 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-ffa77568-cab5-4092-8abd-6063add76f40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:09.987144 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa77568-cab5-4092-8abd-6063add76f40 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:09.991816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f627102-d88b-4ce9-ae92-8dd972aabeaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:09.992460 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f627102-d88b-4ce9-ae92-8dd972aabeaf service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:09.992734 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f627102-d88b-4ce9-ae92-8dd972aabeaf service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:09.992997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f627102-d88b-4ce9-ae92-8dd972aabeaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:10.003560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f627102-d88b-4ce9-ae92-8dd972aabeaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:10.005047 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f627102-d88b-4ce9-ae92-8dd972aabeaf service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:10.015750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1272/2560] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:40:09 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:40:10.476096 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff048f0-a918-49d5-9f32-19f9033a8f59 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:10.487901 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff048f0-a918-49d5-9f32-19f9033a8f59 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:10.491657 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb98707-9806-43ec-8473-71e0aa19b68f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:10.492273 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb98707-9806-43ec-8473-71e0aa19b68f service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:10.492547 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb98707-9806-43ec-8473-71e0aa19b68f service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:10.492813 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb98707-9806-43ec-8473-71e0aa19b68f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:10.513190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:10.553140 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb98707-9806-43ec-8473-71e0aa19b68f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:10.554665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb98707-9806-43ec-8473-71e0aa19b68f service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:10.564642 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1289/2561] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:40:10 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:40:11.493341 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f627102-d88b-4ce9-ae92-8dd972aabeaf service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:11.494993 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-8f627102-d88b-4ce9-ae92-8dd972aabeaf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:11.564497 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f627102-d88b-4ce9-ae92-8dd972aabeaf service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:11.569284 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3412048-2b6a-4a1b-8b8d-6dd3de46d5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:11.569911 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3412048-2b6a-4a1b-8b8d-6dd3de46d5ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:11.570189 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3412048-2b6a-4a1b-8b8d-6dd3de46d5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:11.570497 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3412048-2b6a-4a1b-8b8d-6dd3de46d5ba admin admin] PATH_INFO: `/v3/users/97eff25204014b698b82c51b6b82f7f5` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:11.571656 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3412048-2b6a-4a1b-8b8d-6dd3de46d5ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=97eff25204014b698b82c51b6b82f7f5)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:11.573645 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3412048-2b6a-4a1b-8b8d-6dd3de46d5ba admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:11.769608 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-d3412048-2b6a-4a1b-8b8d-6dd3de46d5ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97eff25204014b698b82c51b6b82f7f5'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:11.835690 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-d3412048-2b6a-4a1b-8b8d-6dd3de46d5ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97eff25204014b698b82c51b6b82f7f5'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:11.836249 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-d3412048-2b6a-4a1b-8b8d-6dd3de46d5ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97eff25204014b698b82c51b6b82f7f5'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:11.836870 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-d3412048-2b6a-4a1b-8b8d-6dd3de46d5ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97eff25204014b698b82c51b6b82f7f5'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:11.850849 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-d3412048-2b6a-4a1b-8b8d-6dd3de46d5ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97eff25204014b698b82c51b6b82f7f5'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:11.864239 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1273/2562] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:40:11 2022] DELETE /identity/v3/users/97eff25204014b698b82c51b6b82f7f5 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:11.868934 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc0305-cab9-4004-98e9-7bf4debd3d5e None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:11.869354 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc0305-cab9-4004-98e9-7bf4debd3d5e None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:11.869567 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc0305-cab9-4004-98e9-7bf4debd3d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:11.888700 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-a3cc0305-cab9-4004-98e9-7bf4debd3d5e None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:11.889907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a3cc0305-cab9-4004-98e9-7bf4debd3d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:11.952651 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1290/2563] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:40:11.962302 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3412048-2b6a-4a1b-8b8d-6dd3de46d5ba admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:11.963762 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-d3412048-2b6a-4a1b-8b8d-6dd3de46d5ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:11.987755 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:12.032683 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3412048-2b6a-4a1b-8b8d-6dd3de46d5ba admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:12.037248 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f45e7af-ead4-47db-8c27-eb6881dbdd0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:12.038053 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f45e7af-ead4-47db-8c27-eb6881dbdd0e service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:12.038332 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f45e7af-ead4-47db-8c27-eb6881dbdd0e service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:12.038632 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f45e7af-ead4-47db-8c27-eb6881dbdd0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:12.049749 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f45e7af-ead4-47db-8c27-eb6881dbdd0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:12.053702 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f45e7af-ead4-47db-8c27-eb6881dbdd0e service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:12.061127 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1274/2564] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:40:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:40:12.456759 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc0305-cab9-4004-98e9-7bf4debd3d5e None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:12.458526 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a3cc0305-cab9-4004-98e9-7bf4debd3d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:12.477938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc0305-cab9-4004-98e9-7bf4debd3d5e None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:12.491614 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9649328-4fb8-42f9-8985-ad748f2ba9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:12.492273 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9649328-4fb8-42f9-8985-ad748f2ba9c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:12.492568 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9649328-4fb8-42f9-8985-ad748f2ba9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:12.492807 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9649328-4fb8-42f9-8985-ad748f2ba9c4 admin admin] PATH_INFO: `/v3/projects/446c6258a94d48bc9d95e29693d79c2d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:12.494023 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9649328-4fb8-42f9-8985-ad748f2ba9c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=446c6258a94d48bc9d95e29693d79c2d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:12.495986 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9649328-4fb8-42f9-8985-ad748f2ba9c4 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:12.629764 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e9649328-4fb8-42f9-8985-ad748f2ba9c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '446c6258a94d48bc9d95e29693d79c2d', 'request_id': 'req-e9649328-4fb8-42f9-8985-ad748f2ba9c4'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:12.734176 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e9649328-4fb8-42f9-8985-ad748f2ba9c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '446c6258a94d48bc9d95e29693d79c2d', 'request_id': 'req-e9649328-4fb8-42f9-8985-ad748f2ba9c4'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:12.740565 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e9649328-4fb8-42f9-8985-ad748f2ba9c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '446c6258a94d48bc9d95e29693d79c2d', 'request_id': 'req-e9649328-4fb8-42f9-8985-ad748f2ba9c4'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:12.752191 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e9649328-4fb8-42f9-8985-ad748f2ba9c4 admin admin] The token cache is being invalidate because project 446c6258a94d48bc9d95e29693d79c2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:40:12.753507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e9649328-4fb8-42f9-8985-ad748f2ba9c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:12.753507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e9649328-4fb8-42f9-8985-ad748f2ba9c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:12.754658 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1291/2565] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:40:12 2022] DELETE /identity/v3/projects/446c6258a94d48bc9d95e29693d79c2d => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:12.758594 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f45e7af-ead4-47db-8c27-eb6881dbdd0e service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:12.768359 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f45e7af-ead4-47db-8c27-eb6881dbdd0e service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:12.829354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3392395-b094-481b-8d24-6ea7e5b8195f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:12.830059 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3392395-b094-481b-8d24-6ea7e5b8195f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:12.830309 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3392395-b094-481b-8d24-6ea7e5b8195f admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:12.830626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3392395-b094-481b-8d24-6ea7e5b8195f admin admin] PATH_INFO: `/v3/users/5801069cc5d54a47aa6fbd7001f4cf6a` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:12.831885 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3392395-b094-481b-8d24-6ea7e5b8195f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5801069cc5d54a47aa6fbd7001f4cf6a)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:12.834181 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3392395-b094-481b-8d24-6ea7e5b8195f admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:13.013969 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-a3392395-b094-481b-8d24-6ea7e5b8195f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5801069cc5d54a47aa6fbd7001f4cf6a'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:13.087006 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-a3392395-b094-481b-8d24-6ea7e5b8195f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5801069cc5d54a47aa6fbd7001f4cf6a'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:13.087686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-a3392395-b094-481b-8d24-6ea7e5b8195f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5801069cc5d54a47aa6fbd7001f4cf6a'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:13.088301 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-a3392395-b094-481b-8d24-6ea7e5b8195f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5801069cc5d54a47aa6fbd7001f4cf6a'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:13.102576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-a3392395-b094-481b-8d24-6ea7e5b8195f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5801069cc5d54a47aa6fbd7001f4cf6a'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:13.130143 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1275/2566] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:40:12 2022] DELETE /identity/v3/users/5801069cc5d54a47aa6fbd7001f4cf6a => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:13.142846 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9649328-4fb8-42f9-8985-ad748f2ba9c4 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:13.145474 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-e9649328-4fb8-42f9-8985-ad748f2ba9c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:13.195599 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:13.225467 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9649328-4fb8-42f9-8985-ad748f2ba9c4 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:13.317330 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47223e-9546-478c-b2c5-5efaf319b350 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:13.318002 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47223e-9546-478c-b2c5-5efaf319b350 service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:13.318277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47223e-9546-478c-b2c5-5efaf319b350 service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:13.318577 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47223e-9546-478c-b2c5-5efaf319b350 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:13.320665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-7d47223e-9546-478c-b2c5-5efaf319b350 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:13.335005 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d47223e-9546-478c-b2c5-5efaf319b350 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:13.336602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d47223e-9546-478c-b2c5-5efaf319b350 service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:13.349554 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1292/2567] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:40:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:40:13.567961 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3392395-b094-481b-8d24-6ea7e5b8195f admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:13.569666 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a3392395-b094-481b-8d24-6ea7e5b8195f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:13.630329 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:13.654691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3392395-b094-481b-8d24-6ea7e5b8195f admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:13.659642 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ffa9aa-090c-4ba7-ae7d-fb301cae9819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:13.660277 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ffa9aa-090c-4ba7-ae7d-fb301cae9819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:13.660555 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ffa9aa-090c-4ba7-ae7d-fb301cae9819 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:13.660840 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ffa9aa-090c-4ba7-ae7d-fb301cae9819 admin admin] PATH_INFO: `/v3/projects/7ee1405efd564d8fb226e5fbb2a2ae41` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:13.661950 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ffa9aa-090c-4ba7-ae7d-fb301cae9819 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ee1405efd564d8fb226e5fbb2a2ae41)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:13.663965 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ffa9aa-090c-4ba7-ae7d-fb301cae9819 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:13.791822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-08ffa9aa-090c-4ba7-ae7d-fb301cae9819 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ee1405efd564d8fb226e5fbb2a2ae41', 'request_id': 'req-08ffa9aa-090c-4ba7-ae7d-fb301cae9819'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:13.798765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-08ffa9aa-090c-4ba7-ae7d-fb301cae9819 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ee1405efd564d8fb226e5fbb2a2ae41', 'request_id': 'req-08ffa9aa-090c-4ba7-ae7d-fb301cae9819'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:13.808595 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-08ffa9aa-090c-4ba7-ae7d-fb301cae9819 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ee1405efd564d8fb226e5fbb2a2ae41', 'request_id': 'req-08ffa9aa-090c-4ba7-ae7d-fb301cae9819'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:13.898634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-08ffa9aa-090c-4ba7-ae7d-fb301cae9819 admin admin] The token cache is being invalidate because project 7ee1405efd564d8fb226e5fbb2a2ae41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:40:13.898932 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-08ffa9aa-090c-4ba7-ae7d-fb301cae9819 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:13.899614 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-08ffa9aa-090c-4ba7-ae7d-fb301cae9819 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:13.904145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1276/2568] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:40:13 2022] DELETE /identity/v3/projects/7ee1405efd564d8fb226e5fbb2a2ae41 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:13.906560 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47223e-9546-478c-b2c5-5efaf319b350 service nova] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:13.916804 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47223e-9546-478c-b2c5-5efaf319b350 service nova] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:13.990016 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae71b6-04a6-45be-954f-9a87d0a127c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:13.990684 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae71b6-04a6-45be-954f-9a87d0a127c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:13.990960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae71b6-04a6-45be-954f-9a87d0a127c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:13.991233 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae71b6-04a6-45be-954f-9a87d0a127c5 admin admin] PATH_INFO: `/v3/users/7fe8456c36af4f6c8d8aab5d5eb5a0ce` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:13.992441 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae71b6-04a6-45be-954f-9a87d0a127c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fe8456c36af4f6c8d8aab5d5eb5a0ce)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:13.994435 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae71b6-04a6-45be-954f-9a87d0a127c5 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:14.192776 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-69ae71b6-04a6-45be-954f-9a87d0a127c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fe8456c36af4f6c8d8aab5d5eb5a0ce'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:14.205312 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-69ae71b6-04a6-45be-954f-9a87d0a127c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fe8456c36af4f6c8d8aab5d5eb5a0ce'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:14.210304 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:14.213332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-69ae71b6-04a6-45be-954f-9a87d0a127c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fe8456c36af4f6c8d8aab5d5eb5a0ce'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:14.315664 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-69ae71b6-04a6-45be-954f-9a87d0a127c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fe8456c36af4f6c8d8aab5d5eb5a0ce'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:14.316224 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-69ae71b6-04a6-45be-954f-9a87d0a127c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fe8456c36af4f6c8d8aab5d5eb5a0ce'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:14.322529 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1293/2569] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:40:13 2022] DELETE /identity/v3/users/7fe8456c36af4f6c8d8aab5d5eb5a0ce => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:14.780601 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ffa9aa-090c-4ba7-ae7d-fb301cae9819 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:14.782050 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-08ffa9aa-090c-4ba7-ae7d-fb301cae9819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:14.801577 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ffa9aa-090c-4ba7-ae7d-fb301cae9819 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:14.861678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9993c7d1-15f1-4939-b680-1b5255df75b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:14.862245 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993c7d1-15f1-4939-b680-1b5255df75b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:14.862543 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993c7d1-15f1-4939-b680-1b5255df75b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:14.862901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993c7d1-15f1-4939-b680-1b5255df75b4 admin admin] PATH_INFO: `/v3/projects/26a23f5b960f4041b13acea131ec6a31` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:14.864037 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9993c7d1-15f1-4939-b680-1b5255df75b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26a23f5b960f4041b13acea131ec6a31)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:14.865913 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9993c7d1-15f1-4939-b680-1b5255df75b4 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:15.018666 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-9993c7d1-15f1-4939-b680-1b5255df75b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26a23f5b960f4041b13acea131ec6a31', 'request_id': 'req-9993c7d1-15f1-4939-b680-1b5255df75b4'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:15.025289 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-9993c7d1-15f1-4939-b680-1b5255df75b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26a23f5b960f4041b13acea131ec6a31', 'request_id': 'req-9993c7d1-15f1-4939-b680-1b5255df75b4'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:15.032407 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-9993c7d1-15f1-4939-b680-1b5255df75b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26a23f5b960f4041b13acea131ec6a31', 'request_id': 'req-9993c7d1-15f1-4939-b680-1b5255df75b4'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:15.140110 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:15.145011 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-9993c7d1-15f1-4939-b680-1b5255df75b4 admin admin] The token cache is being invalidate because project 26a23f5b960f4041b13acea131ec6a31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:40:15.149401 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-9993c7d1-15f1-4939-b680-1b5255df75b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:15.149401 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-9993c7d1-15f1-4939-b680-1b5255df75b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:15.149401 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1277/2570] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:40:14 2022] DELETE /identity/v3/projects/26a23f5b960f4041b13acea131ec6a31 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:15.154848 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6223e97-f216-47b3-b8ba-dfeca4509fc3 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:15.155307 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6223e97-f216-47b3-b8ba-dfeca4509fc3 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:15.155581 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6223e97-f216-47b3-b8ba-dfeca4509fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:15.175331 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-d6223e97-f216-47b3-b8ba-dfeca4509fc3 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:15.176529 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-d6223e97-f216-47b3-b8ba-dfeca4509fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:15.236021 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1294/2571] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:40:15.244465 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d292ded-24b3-4b29-a9ca-34e50b375685 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:15.244740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d292ded-24b3-4b29-a9ca-34e50b375685 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:15.245007 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d292ded-24b3-4b29-a9ca-34e50b375685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:15.265221 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-0d292ded-24b3-4b29-a9ca-34e50b375685 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:15.266453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-0d292ded-24b3-4b29-a9ca-34e50b375685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:15.274127 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1278/2572] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:40:15.278528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6223e97-f216-47b3-b8ba-dfeca4509fc3 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:15.288731 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6223e97-f216-47b3-b8ba-dfeca4509fc3 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:15.293271 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe17a38b-7edd-4e90-a43b-bb64ee9c6c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:15.293861 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17a38b-7edd-4e90-a43b-bb64ee9c6c0b admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:15.294135 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17a38b-7edd-4e90-a43b-bb64ee9c6c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:15.294572 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17a38b-7edd-4e90-a43b-bb64ee9c6c0b admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:15.295217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe17a38b-7edd-4e90-a43b-bb64ee9c6c0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:40:15.295516 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe17a38b-7edd-4e90-a43b-bb64ee9c6c0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:15.297100 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe17a38b-7edd-4e90-a43b-bb64ee9c6c0b admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:15.308263 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1295/2573] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:40:15.312553 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d292ded-24b3-4b29-a9ca-34e50b375685 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:15.322074 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d292ded-24b3-4b29-a9ca-34e50b375685 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:15.327908 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1571e-8bbb-41df-8a85-37959e264058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:15.328515 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1571e-8bbb-41df-8a85-37959e264058 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:15.328792 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1571e-8bbb-41df-8a85-37959e264058 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:15.329062 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1571e-8bbb-41df-8a85-37959e264058 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:15.329698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1571e-8bbb-41df-8a85-37959e264058 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:15.331626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1571e-8bbb-41df-8a85-37959e264058 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:15.439137 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1279/2574] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:40:15 2022] POST /identity/v3/projects => generated 393 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:40:15.442783 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe17a38b-7edd-4e90-a43b-bb64ee9c6c0b admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:15.454573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe17a38b-7edd-4e90-a43b-bb64ee9c6c0b admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:15.512503 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65350b-22e2-4c9f-be41-05c8be2677c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:15.513101 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65350b-22e2-4c9f-be41-05c8be2677c9 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:15.513376 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65350b-22e2-4c9f-be41-05c8be2677c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:15.513644 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65350b-22e2-4c9f-be41-05c8be2677c9 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:15.514332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d65350b-22e2-4c9f-be41-05c8be2677c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:15.516907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d65350b-22e2-4c9f-be41-05c8be2677c9 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:15.595995 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1296/2575] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:40:15 2022] POST /identity/v3/users => generated 297 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:40:15.600135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1571e-8bbb-41df-8a85-37959e264058 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:15.610868 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1571e-8bbb-41df-8a85-37959e264058 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:15.615721 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19495bc-6983-4557-88a8-1b6aa7e5d882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:15.616321 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19495bc-6983-4557-88a8-1b6aa7e5d882 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:15.616595 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19495bc-6983-4557-88a8-1b6aa7e5d882 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:15.616906 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19495bc-6983-4557-88a8-1b6aa7e5d882 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:15.617638 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19495bc-6983-4557-88a8-1b6aa7e5d882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:15.619240 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19495bc-6983-4557-88a8-1b6aa7e5d882 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:15.630168 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1280/2576] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:40:15.634468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65350b-22e2-4c9f-be41-05c8be2677c9 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:15.645331 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65350b-22e2-4c9f-be41-05c8be2677c9 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:15.650374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2b65d-8fc3-4270-bf6e-f14eaf2c742c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:15.651003 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2b65d-8fc3-4270-bf6e-f14eaf2c742c admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:15.651281 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2b65d-8fc3-4270-bf6e-f14eaf2c742c admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:15.651553 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2b65d-8fc3-4270-bf6e-f14eaf2c742c admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:15.652192 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df2b65d-8fc3-4270-bf6e-f14eaf2c742c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:15.653786 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df2b65d-8fc3-4270-bf6e-f14eaf2c742c admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:15.665000 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1297/2577] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:40:15.669305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19495bc-6983-4557-88a8-1b6aa7e5d882 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:15.679286 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19495bc-6983-4557-88a8-1b6aa7e5d882 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:15.683839 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecc383-b7c4-441f-93cb-2035b3a057c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:15.684467 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecc383-b7c4-441f-93cb-2035b3a057c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:15.684738 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecc383-b7c4-441f-93cb-2035b3a057c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:15.685004 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecc383-b7c4-441f-93cb-2035b3a057c5 admin admin] PATH_INFO: `/v3/projects/11acc7ea998145968df2bc88578a97d2/users/35d51afc29a04961a4c2c302dc0e0c50/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:15.699859 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ecc383-b7c4-441f-93cb-2035b3a057c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11acc7ea998145968df2bc88578a97d2, user_id=35d51afc29a04961a4c2c302dc0e0c50, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:15.703135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ecc383-b7c4-441f-93cb-2035b3a057c5 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:15.778852 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1281/2578] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:40:15 2022] PUT /identity/v3/projects/11acc7ea998145968df2bc88578a97d2/users/35d51afc29a04961a4c2c302dc0e0c50/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:15.791602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65cf7c2-5bff-4d1b-a72b-4544e9bd00c5 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:15.791884 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65cf7c2-5bff-4d1b-a72b-4544e9bd00c5 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:15.792183 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65cf7c2-5bff-4d1b-a72b-4544e9bd00c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:15.821864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-e65cf7c2-5bff-4d1b-a72b-4544e9bd00c5 None None] MFA Rules not processed for user `35d51afc29a04961a4c2c302dc0e0c50`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:15.823257 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-e65cf7c2-5bff-4d1b-a72b-4544e9bd00c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:15.865613 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:15.888756 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1298/2579] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:40:15.893402 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecc383-b7c4-441f-93cb-2035b3a057c5 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:15.958935 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecc383-b7c4-441f-93cb-2035b3a057c5 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:15.964151 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c9c58-ff03-4545-bebb-a6a32c5f2977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:15.964779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c9c58-ff03-4545-bebb-a6a32c5f2977 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:15.965057 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c9c58-ff03-4545-bebb-a6a32c5f2977 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:15.965327 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c9c58-ff03-4545-bebb-a6a32c5f2977 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:15.965954 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9c9c58-ff03-4545-bebb-a6a32c5f2977 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:15.968209 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9c9c58-ff03-4545-bebb-a6a32c5f2977 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:16.060167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1282/2580] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:40:15 2022] POST /identity/v3/projects => generated 393 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:40:16.065191 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65cf7c2-5bff-4d1b-a72b-4544e9bd00c5 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:16.076053 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65cf7c2-5bff-4d1b-a72b-4544e9bd00c5 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:16.139251 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0a989-4888-437d-8dc2-b7bc1a8e7c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:16.139834 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0a989-4888-437d-8dc2-b7bc1a8e7c04 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:16.140216 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0a989-4888-437d-8dc2-b7bc1a8e7c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:16.140495 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0a989-4888-437d-8dc2-b7bc1a8e7c04 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:16.141144 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0a989-4888-437d-8dc2-b7bc1a8e7c04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:16.143163 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0a989-4888-437d-8dc2-b7bc1a8e7c04 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:16.229846 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1299/2581] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:40:16 2022] POST /identity/v3/users => generated 303 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:40:16.235152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c9c58-ff03-4545-bebb-a6a32c5f2977 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:16.246066 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c9c58-ff03-4545-bebb-a6a32c5f2977 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:16.250833 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea57d9-1676-43d1-8f31-51d658501b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:16.251471 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea57d9-1676-43d1-8f31-51d658501b31 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:16.251746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea57d9-1676-43d1-8f31-51d658501b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:16.252019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea57d9-1676-43d1-8f31-51d658501b31 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:16.252679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea57d9-1676-43d1-8f31-51d658501b31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:16.254274 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea57d9-1676-43d1-8f31-51d658501b31 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:16.266307 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1283/2582] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:40:16.270305 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0a989-4888-437d-8dc2-b7bc1a8e7c04 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:16.280926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0a989-4888-437d-8dc2-b7bc1a8e7c04 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:16.285783 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180db2a-89c3-4442-aab5-1cc918b922dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:16.286273 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180db2a-89c3-4442-aab5-1cc918b922dd admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:16.286578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180db2a-89c3-4442-aab5-1cc918b922dd admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:16.286846 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180db2a-89c3-4442-aab5-1cc918b922dd admin admin] PATH_INFO: `/v3/projects/c83bcdfa918e45c69d41684ecea62118/users/5bf9bf7a24ca432087e568e5e171568a/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:16.306721 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c180db2a-89c3-4442-aab5-1cc918b922dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c83bcdfa918e45c69d41684ecea62118, user_id=5bf9bf7a24ca432087e568e5e171568a, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:16.309514 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c180db2a-89c3-4442-aab5-1cc918b922dd admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:16.372615 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1300/2583] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:40:16 2022] PUT /identity/v3/projects/c83bcdfa918e45c69d41684ecea62118/users/5bf9bf7a24ca432087e568e5e171568a/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:16.376529 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea57d9-1676-43d1-8f31-51d658501b31 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:16.386484 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea57d9-1676-43d1-8f31-51d658501b31 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:16.444783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576a0a6-4a6b-4316-a347-4b4680923bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:16.445450 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576a0a6-4a6b-4316-a347-4b4680923bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:16.445723 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576a0a6-4a6b-4316-a347-4b4680923bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:16.445988 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576a0a6-4a6b-4316-a347-4b4680923bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:16.446650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1576a0a6-4a6b-4316-a347-4b4680923bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:16.448242 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1576a0a6-4a6b-4316-a347-4b4680923bf0 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:16.459218 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1284/2584] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:40:16.463394 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180db2a-89c3-4442-aab5-1cc918b922dd admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:16.474717 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180db2a-89c3-4442-aab5-1cc918b922dd admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:16.479327 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29712dea-ec34-4200-9cb7-928b0b9f996a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:16.479973 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29712dea-ec34-4200-9cb7-928b0b9f996a admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:16.480248 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29712dea-ec34-4200-9cb7-928b0b9f996a admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:16.480518 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29712dea-ec34-4200-9cb7-928b0b9f996a admin admin] PATH_INFO: `/v3/projects/c83bcdfa918e45c69d41684ecea62118/users/5bf9bf7a24ca432087e568e5e171568a/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:16.482567 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29712dea-ec34-4200-9cb7-928b0b9f996a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c83bcdfa918e45c69d41684ecea62118, user_id=5bf9bf7a24ca432087e568e5e171568a, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:16.485884 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29712dea-ec34-4200-9cb7-928b0b9f996a admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:16.496763 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1301/2585] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:40:16 2022] PUT /identity/v3/projects/c83bcdfa918e45c69d41684ecea62118/users/5bf9bf7a24ca432087e568e5e171568a/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:16.506882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622da1e-c200-4228-b909-e451a3d8ecdb None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:16.507164 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622da1e-c200-4228-b909-e451a3d8ecdb None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:16.507433 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622da1e-c200-4228-b909-e451a3d8ecdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:16.516121 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:16.536765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-6622da1e-c200-4228-b909-e451a3d8ecdb None None] MFA Rules not processed for user `5bf9bf7a24ca432087e568e5e171568a`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:16.537973 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-6622da1e-c200-4228-b909-e451a3d8ecdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:16.608648 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1285/2586] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:16 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:40:16.620977 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29712dea-ec34-4200-9cb7-928b0b9f996a admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:16.622836 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-29712dea-ec34-4200-9cb7-928b0b9f996a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:16.684198 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29712dea-ec34-4200-9cb7-928b0b9f996a admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:16.687948 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9afa7-4a8b-4212-9d58-9cefe5a11cde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:16.688548 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9afa7-4a8b-4212-9d58-9cefe5a11cde service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:16.688824 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9afa7-4a8b-4212-9d58-9cefe5a11cde service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:16.689093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9afa7-4a8b-4212-9d58-9cefe5a11cde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:16.705797 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf9afa7-4a8b-4212-9d58-9cefe5a11cde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:16.705797 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf9afa7-4a8b-4212-9d58-9cefe5a11cde service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:16.709806 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1302/2587] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:40:16 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:40:16.758204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622da1e-c200-4228-b909-e451a3d8ecdb None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:16.759605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-6622da1e-c200-4228-b909-e451a3d8ecdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:16.772487 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622da1e-c200-4228-b909-e451a3d8ecdb None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:16.776112 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3681841-c0e2-41d9-b8db-40c2239dcbff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:16.776736 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3681841-c0e2-41d9-b8db-40c2239dcbff service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:16.777013 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3681841-c0e2-41d9-b8db-40c2239dcbff service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:16.777282 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3681841-c0e2-41d9-b8db-40c2239dcbff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:16.835615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3681841-c0e2-41d9-b8db-40c2239dcbff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:16.837073 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3681841-c0e2-41d9-b8db-40c2239dcbff service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:16.846148 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1286/2588] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:40:16 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:40:24.548447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9afa7-4a8b-4212-9d58-9cefe5a11cde service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:24.554138 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-baf9afa7-4a8b-4212-9d58-9cefe5a11cde service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:24.598450 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:24.651296 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9afa7-4a8b-4212-9d58-9cefe5a11cde service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:24.659080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9dad4-2905-42f4-9c70-f3f7e47cb06d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:24.659720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9dad4-2905-42f4-9c70-f3f7e47cb06d service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:24.659993 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9dad4-2905-42f4-9c70-f3f7e47cb06d service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:24.665323 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9dad4-2905-42f4-9c70-f3f7e47cb06d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:24.665323 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-fad9dad4-2905-42f4-9c70-f3f7e47cb06d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:24.675235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad9dad4-2905-42f4-9c70-f3f7e47cb06d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:24.676922 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad9dad4-2905-42f4-9c70-f3f7e47cb06d service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:24.690470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1303/2589] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:40:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:40:25.978666 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3681841-c0e2-41d9-b8db-40c2239dcbff service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:25.980175 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-c3681841-c0e2-41d9-b8db-40c2239dcbff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:26.047976 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3681841-c0e2-41d9-b8db-40c2239dcbff service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:26.052478 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9638527-18bd-4473-a3c8-81ce253f6df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:26.053101 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9638527-18bd-4473-a3c8-81ce253f6df6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:26.053371 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9638527-18bd-4473-a3c8-81ce253f6df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:26.053630 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9638527-18bd-4473-a3c8-81ce253f6df6 admin admin] PATH_INFO: `/v3/users/28f9e182f32b48bd84bdd94f28884f4c` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:26.054789 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9638527-18bd-4473-a3c8-81ce253f6df6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28f9e182f32b48bd84bdd94f28884f4c)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:26.056705 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9638527-18bd-4473-a3c8-81ce253f6df6 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:26.133018 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9dad4-2905-42f4-9c70-f3f7e47cb06d service nova] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:26.144091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9dad4-2905-42f4-9c70-f3f7e47cb06d service nova] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:26.148735 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7561930-f19b-4d19-95ba-72149cbd56f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:26.149341 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7561930-f19b-4d19-95ba-72149cbd56f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:26.149613 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7561930-f19b-4d19-95ba-72149cbd56f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:26.149881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7561930-f19b-4d19-95ba-72149cbd56f7 admin admin] PATH_INFO: `/v3/users/35d51afc29a04961a4c2c302dc0e0c50` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:26.150994 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7561930-f19b-4d19-95ba-72149cbd56f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35d51afc29a04961a4c2c302dc0e0c50)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:26.152951 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7561930-f19b-4d19-95ba-72149cbd56f7 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:26.274567 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:26.289754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-b9638527-18bd-4473-a3c8-81ce253f6df6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28f9e182f32b48bd84bdd94f28884f4c'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:26.359288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-b9638527-18bd-4473-a3c8-81ce253f6df6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28f9e182f32b48bd84bdd94f28884f4c'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:26.359907 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-b9638527-18bd-4473-a3c8-81ce253f6df6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28f9e182f32b48bd84bdd94f28884f4c'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:26.360595 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-b9638527-18bd-4473-a3c8-81ce253f6df6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28f9e182f32b48bd84bdd94f28884f4c'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:26.373537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-b7561930-f19b-4d19-95ba-72149cbd56f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35d51afc29a04961a4c2c302dc0e0c50'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:26.373891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-b9638527-18bd-4473-a3c8-81ce253f6df6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28f9e182f32b48bd84bdd94f28884f4c'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:26.386282 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-b7561930-f19b-4d19-95ba-72149cbd56f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35d51afc29a04961a4c2c302dc0e0c50'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:26.387084 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1287/2590] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:40:25 2022] DELETE /identity/v3/users/28f9e182f32b48bd84bdd94f28884f4c => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:26.392179 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d7c19-be60-4b37-9e72-2e2cc21088bc None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:26.392480 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d7c19-be60-4b37-9e72-2e2cc21088bc None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:26.392773 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d7c19-be60-4b37-9e72-2e2cc21088bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:26.398487 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-b7561930-f19b-4d19-95ba-72149cbd56f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35d51afc29a04961a4c2c302dc0e0c50'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:26.412394 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-497d7c19-be60-4b37-9e72-2e2cc21088bc None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:26.413604 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-497d7c19-be60-4b37-9e72-2e2cc21088bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:26.470093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-b7561930-f19b-4d19-95ba-72149cbd56f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35d51afc29a04961a4c2c302dc0e0c50'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:26.471807 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-b7561930-f19b-4d19-95ba-72149cbd56f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35d51afc29a04961a4c2c302dc0e0c50'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:26.473387 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1288/2591] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:40:26.483126 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d7c19-be60-4b37-9e72-2e2cc21088bc None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:26.483977 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1304/2592] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:40:26 2022] DELETE /identity/v3/users/35d51afc29a04961a4c2c302dc0e0c50 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:40:26.493109 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-497d7c19-be60-4b37-9e72-2e2cc21088bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:26.493230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba27624-2e47-4620-a924-99329a25a83d None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:26.493230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba27624-2e47-4620-a924-99329a25a83d None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:26.493230 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba27624-2e47-4620-a924-99329a25a83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:26.510733 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-fba27624-2e47-4620-a924-99329a25a83d None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:26.512089 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-fba27624-2e47-4620-a924-99329a25a83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:26.547047 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d7c19-be60-4b37-9e72-2e2cc21088bc None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:26.576951 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1305/2593] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:40:26.587651 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba27624-2e47-4620-a924-99329a25a83d None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:26.589067 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-fba27624-2e47-4620-a924-99329a25a83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:26.601921 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b91510-ba89-4d43-80b8-bd21157f779f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:26.601921 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b91510-ba89-4d43-80b8-bd21157f779f service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:26.601921 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b91510-ba89-4d43-80b8-bd21157f779f service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:26.601921 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b91510-ba89-4d43-80b8-bd21157f779f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:26.603495 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-17b91510-ba89-4d43-80b8-bd21157f779f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:26.615832 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b91510-ba89-4d43-80b8-bd21157f779f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:26.617353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b91510-ba89-4d43-80b8-bd21157f779f service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:26.618058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:26.627635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1289/2594] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:40:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:40:26.647281 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba27624-2e47-4620-a924-99329a25a83d None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:26.650912 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668eb5d2-4276-467a-a4e9-c7d458bdb420 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:26.651513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668eb5d2-4276-467a-a4e9-c7d458bdb420 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:26.651784 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668eb5d2-4276-467a-a4e9-c7d458bdb420 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:26.652055 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668eb5d2-4276-467a-a4e9-c7d458bdb420 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:26.661483 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668eb5d2-4276-467a-a4e9-c7d458bdb420 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:26.663030 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668eb5d2-4276-467a-a4e9-c7d458bdb420 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:26.672996 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1306/2595] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:40:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:40:27.152391 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b91510-ba89-4d43-80b8-bd21157f779f service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:27.152851 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-17b91510-ba89-4d43-80b8-bd21157f779f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:27.165302 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b91510-ba89-4d43-80b8-bd21157f779f service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:27.169957 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c5033-f957-4f49-b32b-ee09c2cc215d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:27.170908 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c5033-f957-4f49-b32b-ee09c2cc215d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:27.175235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c5033-f957-4f49-b32b-ee09c2cc215d admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:27.175235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c5033-f957-4f49-b32b-ee09c2cc215d admin admin] PATH_INFO: `/v3/projects/a1dcff769e8d44909c60cbdb920d253e` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:27.175235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163c5033-f957-4f49-b32b-ee09c2cc215d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1dcff769e8d44909c60cbdb920d253e)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:27.175235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163c5033-f957-4f49-b32b-ee09c2cc215d admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:27.176762 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:27.182092 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668eb5d2-4276-467a-a4e9-c7d458bdb420 service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:27.184771 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-668eb5d2-4276-467a-a4e9-c7d458bdb420 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:27.203000 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668eb5d2-4276-467a-a4e9-c7d458bdb420 service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:27.210165 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05680650-22e4-4545-abd2-3ef8c2c7fb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:27.211030 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05680650-22e4-4545-abd2-3ef8c2c7fb66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:27.211403 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05680650-22e4-4545-abd2-3ef8c2c7fb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:27.211744 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05680650-22e4-4545-abd2-3ef8c2c7fb66 admin admin] PATH_INFO: `/v3/projects/11acc7ea998145968df2bc88578a97d2` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:27.213210 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05680650-22e4-4545-abd2-3ef8c2c7fb66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11acc7ea998145968df2bc88578a97d2)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:27.215239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05680650-22e4-4545-abd2-3ef8c2c7fb66 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:27.435313 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-05680650-22e4-4545-abd2-3ef8c2c7fb66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11acc7ea998145968df2bc88578a97d2', 'request_id': 'req-05680650-22e4-4545-abd2-3ef8c2c7fb66'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:27.436387 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-163c5033-f957-4f49-b32b-ee09c2cc215d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1dcff769e8d44909c60cbdb920d253e', 'request_id': 'req-163c5033-f957-4f49-b32b-ee09c2cc215d'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:27.443512 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-163c5033-f957-4f49-b32b-ee09c2cc215d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1dcff769e8d44909c60cbdb920d253e', 'request_id': 'req-163c5033-f957-4f49-b32b-ee09c2cc215d'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:27.449024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-163c5033-f957-4f49-b32b-ee09c2cc215d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1dcff769e8d44909c60cbdb920d253e', 'request_id': 'req-163c5033-f957-4f49-b32b-ee09c2cc215d'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:27.532877 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-05680650-22e4-4545-abd2-3ef8c2c7fb66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11acc7ea998145968df2bc88578a97d2', 'request_id': 'req-05680650-22e4-4545-abd2-3ef8c2c7fb66'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:27.539312 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-05680650-22e4-4545-abd2-3ef8c2c7fb66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11acc7ea998145968df2bc88578a97d2', 'request_id': 'req-05680650-22e4-4545-abd2-3ef8c2c7fb66'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:27.551251 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-05680650-22e4-4545-abd2-3ef8c2c7fb66 admin admin] The token cache is being invalidate because project 11acc7ea998145968df2bc88578a97d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:40:27.564073 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-05680650-22e4-4545-abd2-3ef8c2c7fb66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:27.564073 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-05680650-22e4-4545-abd2-3ef8c2c7fb66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:27.564073 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1307/2596] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:40:27 2022] DELETE /identity/v3/projects/11acc7ea998145968df2bc88578a97d2 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:40:27.564073 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05680650-22e4-4545-abd2-3ef8c2c7fb66 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:27.564073 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-05680650-22e4-4545-abd2-3ef8c2c7fb66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:27.622959 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-163c5033-f957-4f49-b32b-ee09c2cc215d admin admin] The token cache is being invalidate because project a1dcff769e8d44909c60cbdb920d253e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:40:27.623247 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-163c5033-f957-4f49-b32b-ee09c2cc215d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:27.623892 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-163c5033-f957-4f49-b32b-ee09c2cc215d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:27.624022 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05680650-22e4-4545-abd2-3ef8c2c7fb66 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:27.625376 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1290/2597] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:40:27 2022] DELETE /identity/v3/projects/a1dcff769e8d44909c60cbdb920d253e => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:27.629584 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c5033-f957-4f49-b32b-ee09c2cc215d admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:27.631002 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-163c5033-f957-4f49-b32b-ee09c2cc215d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:27.676791 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:27.683602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e766f-1c93-4d14-b4b6-2d01210bf85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:27.684275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e766f-1c93-4d14-b4b6-2d01210bf85f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:27.684873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e766f-1c93-4d14-b4b6-2d01210bf85f admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:27.684873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e766f-1c93-4d14-b4b6-2d01210bf85f admin admin] PATH_INFO: `/v3/users/5bf9bf7a24ca432087e568e5e171568a` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:27.686130 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e766f-1c93-4d14-b4b6-2d01210bf85f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bf9bf7a24ca432087e568e5e171568a)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:27.688132 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e766f-1c93-4d14-b4b6-2d01210bf85f admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:27.696706 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c5033-f957-4f49-b32b-ee09c2cc215d admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:27.701485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73659833-7fac-48df-adc8-aadc8ee0c2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:27.702107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73659833-7fac-48df-adc8-aadc8ee0c2c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:27.702383 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73659833-7fac-48df-adc8-aadc8ee0c2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:27.702676 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73659833-7fac-48df-adc8-aadc8ee0c2c4 admin admin] PATH_INFO: `/v3/users/ff7c35457c3e43d1b912132785016789` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:27.704754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73659833-7fac-48df-adc8-aadc8ee0c2c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff7c35457c3e43d1b912132785016789)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:27.706694 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73659833-7fac-48df-adc8-aadc8ee0c2c4 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:27.928221 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-df4e766f-1c93-4d14-b4b6-2d01210bf85f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bf9bf7a24ca432087e568e5e171568a'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:27.940052 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-df4e766f-1c93-4d14-b4b6-2d01210bf85f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bf9bf7a24ca432087e568e5e171568a'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:27.947793 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-df4e766f-1c93-4d14-b4b6-2d01210bf85f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bf9bf7a24ca432087e568e5e171568a'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:27.957547 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-73659833-7fac-48df-adc8-aadc8ee0c2c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff7c35457c3e43d1b912132785016789'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:28.029525 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-df4e766f-1c93-4d14-b4b6-2d01210bf85f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bf9bf7a24ca432087e568e5e171568a'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:28.030328 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-df4e766f-1c93-4d14-b4b6-2d01210bf85f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bf9bf7a24ca432087e568e5e171568a'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:28.039771 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1308/2598] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:40:27 2022] DELETE /identity/v3/users/5bf9bf7a24ca432087e568e5e171568a => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:40:28.114763 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-73659833-7fac-48df-adc8-aadc8ee0c2c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff7c35457c3e43d1b912132785016789'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:28.118949 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-73659833-7fac-48df-adc8-aadc8ee0c2c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff7c35457c3e43d1b912132785016789'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:28.118949 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-73659833-7fac-48df-adc8-aadc8ee0c2c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff7c35457c3e43d1b912132785016789'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:28.130299 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-73659833-7fac-48df-adc8-aadc8ee0c2c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff7c35457c3e43d1b912132785016789'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:28.143562 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1291/2599] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:40:27 2022] DELETE /identity/v3/users/ff7c35457c3e43d1b912132785016789 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:40:28.535112 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e766f-1c93-4d14-b4b6-2d01210bf85f admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:28.536698 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-df4e766f-1c93-4d14-b4b6-2d01210bf85f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:28.605913 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e766f-1c93-4d14-b4b6-2d01210bf85f admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:28.663101 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd844192-3c45-45d5-8706-5ba31e59702b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:28.663608 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd844192-3c45-45d5-8706-5ba31e59702b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:28.663885 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd844192-3c45-45d5-8706-5ba31e59702b admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:28.664157 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd844192-3c45-45d5-8706-5ba31e59702b admin admin] PATH_INFO: `/v3/projects/c83bcdfa918e45c69d41684ecea62118` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:28.665412 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd844192-3c45-45d5-8706-5ba31e59702b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c83bcdfa918e45c69d41684ecea62118)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:28.671158 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd844192-3c45-45d5-8706-5ba31e59702b admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:28.671289 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73659833-7fac-48df-adc8-aadc8ee0c2c4 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:28.671801 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-73659833-7fac-48df-adc8-aadc8ee0c2c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:28.698675 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73659833-7fac-48df-adc8-aadc8ee0c2c4 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:28.703539 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d95ca5-de7b-4e17-b89e-f27c83beb914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:28.704886 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d95ca5-de7b-4e17-b89e-f27c83beb914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:28.705286 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d95ca5-de7b-4e17-b89e-f27c83beb914 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:28.705629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d95ca5-de7b-4e17-b89e-f27c83beb914 admin admin] PATH_INFO: `/v3/projects/471a719670534b67942fdec710b1a6a9` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:28.707119 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d95ca5-de7b-4e17-b89e-f27c83beb914 admin admin] RBAC: Authorizing `identity:delete_project(project_id=471a719670534b67942fdec710b1a6a9)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:28.709018 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d95ca5-de7b-4e17-b89e-f27c83beb914 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:28.713251 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:28.803745 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:28.822837 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cd844192-3c45-45d5-8706-5ba31e59702b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c83bcdfa918e45c69d41684ecea62118', 'request_id': 'req-cd844192-3c45-45d5-8706-5ba31e59702b'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:28.825160 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-b3d95ca5-de7b-4e17-b89e-f27c83beb914 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '471a719670534b67942fdec710b1a6a9', 'request_id': 'req-b3d95ca5-de7b-4e17-b89e-f27c83beb914'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:28.832453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-b3d95ca5-de7b-4e17-b89e-f27c83beb914 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '471a719670534b67942fdec710b1a6a9', 'request_id': 'req-b3d95ca5-de7b-4e17-b89e-f27c83beb914'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:28.838273 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-b3d95ca5-de7b-4e17-b89e-f27c83beb914 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '471a719670534b67942fdec710b1a6a9', 'request_id': 'req-b3d95ca5-de7b-4e17-b89e-f27c83beb914'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:28.891134 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cd844192-3c45-45d5-8706-5ba31e59702b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c83bcdfa918e45c69d41684ecea62118', 'request_id': 'req-cd844192-3c45-45d5-8706-5ba31e59702b'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:28.897789 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cd844192-3c45-45d5-8706-5ba31e59702b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c83bcdfa918e45c69d41684ecea62118', 'request_id': 'req-cd844192-3c45-45d5-8706-5ba31e59702b'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:28.908898 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cd844192-3c45-45d5-8706-5ba31e59702b admin admin] The token cache is being invalidate because project c83bcdfa918e45c69d41684ecea62118 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:40:28.909182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cd844192-3c45-45d5-8706-5ba31e59702b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:28.909815 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cd844192-3c45-45d5-8706-5ba31e59702b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:28.911331 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1309/2600] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:40:28 2022] DELETE /identity/v3/projects/c83bcdfa918e45c69d41684ecea62118 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:28.915935 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a312664-5846-4f51-8789-50b73e72ab97 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:28.916212 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a312664-5846-4f51-8789-50b73e72ab97 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:28.916484 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a312664-5846-4f51-8789-50b73e72ab97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:28.935460 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-2a312664-5846-4f51-8789-50b73e72ab97 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:28.936659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2a312664-5846-4f51-8789-50b73e72ab97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:28.995166 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1310/2601] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:40:28.999945 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-b3d95ca5-de7b-4e17-b89e-f27c83beb914 admin admin] The token cache is being invalidate because project 471a719670534b67942fdec710b1a6a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:40:29.000234 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-b3d95ca5-de7b-4e17-b89e-f27c83beb914 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:29.001536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-b3d95ca5-de7b-4e17-b89e-f27c83beb914 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:29.003375 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1292/2602] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:40:28 2022] DELETE /identity/v3/projects/471a719670534b67942fdec710b1a6a9 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:40:29.003633 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4676ab6-7e81-43e0-9e44-b15c3b28a1c9 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:29.003921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4676ab6-7e81-43e0-9e44-b15c3b28a1c9 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:29.004200 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4676ab6-7e81-43e0-9e44-b15c3b28a1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:29.007661 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5010b-1943-49f4-8f10-f2e50732c836 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:29.007970 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5010b-1943-49f4-8f10-f2e50732c836 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:29.008245 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5010b-1943-49f4-8f10-f2e50732c836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:29.024661 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-b4676ab6-7e81-43e0-9e44-b15c3b28a1c9 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:29.025881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-b4676ab6-7e81-43e0-9e44-b15c3b28a1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:29.028956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-b4d5010b-1943-49f4-8f10-f2e50732c836 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:29.030210 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-b4d5010b-1943-49f4-8f10-f2e50732c836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:29.037054 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1311/2603] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:40:29.041199 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1293/2604] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:40:29.042284 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4676ab6-7e81-43e0-9e44-b15c3b28a1c9 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:29.049085 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30733fe1-5991-4b66-bb53-9f803888e1f0 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:29.049471 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30733fe1-5991-4b66-bb53-9f803888e1f0 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:29.049752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30733fe1-5991-4b66-bb53-9f803888e1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:29.052017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4676ab6-7e81-43e0-9e44-b15c3b28a1c9 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:29.056739 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc8e6f2-32ec-448e-a692-f2c62b31823f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:29.057276 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8e6f2-32ec-448e-a692-f2c62b31823f admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:29.057702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8e6f2-32ec-448e-a692-f2c62b31823f admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:29.057818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8e6f2-32ec-448e-a692-f2c62b31823f admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:29.059264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc8e6f2-32ec-448e-a692-f2c62b31823f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:40:29.059264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc8e6f2-32ec-448e-a692-f2c62b31823f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:29.060331 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc8e6f2-32ec-448e-a692-f2c62b31823f admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:29.070231 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-30733fe1-5991-4b66-bb53-9f803888e1f0 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:29.071486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-30733fe1-5991-4b66-bb53-9f803888e1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:29.073719 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1312/2605] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:40:29.077655 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc8e6f2-32ec-448e-a692-f2c62b31823f admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:29.083956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1294/2606] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:40:29.088276 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30733fe1-5991-4b66-bb53-9f803888e1f0 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:29.088513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc8e6f2-32ec-448e-a692-f2c62b31823f admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:29.093351 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14817ea-1b8d-460a-beb7-b57d747c63ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:29.093960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14817ea-1b8d-460a-beb7-b57d747c63ac admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:29.094280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14817ea-1b8d-460a-beb7-b57d747c63ac admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:29.094677 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14817ea-1b8d-460a-beb7-b57d747c63ac admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:29.095576 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14817ea-1b8d-460a-beb7-b57d747c63ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:29.097661 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14817ea-1b8d-460a-beb7-b57d747c63ac admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:29.098795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30733fe1-5991-4b66-bb53-9f803888e1f0 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:29.104000 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354a150-6646-4d11-b6a4-f2393b9f7ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:29.104601 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354a150-6646-4d11-b6a4-f2393b9f7ece admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:29.104872 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354a150-6646-4d11-b6a4-f2393b9f7ece admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:29.110598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354a150-6646-4d11-b6a4-f2393b9f7ece admin admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:29.110598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f354a150-6646-4d11-b6a4-f2393b9f7ece admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:40:29.110598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f354a150-6646-4d11-b6a4-f2393b9f7ece admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:29.110598 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f354a150-6646-4d11-b6a4-f2393b9f7ece admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:29.120733 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1295/2607] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:40:29.125082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354a150-6646-4d11-b6a4-f2393b9f7ece admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:29.136081 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354a150-6646-4d11-b6a4-f2393b9f7ece admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:29.140805 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f1091-4450-46e7-b132-1dd227a60210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:29.141548 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f1091-4450-46e7-b132-1dd227a60210 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:29.142036 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f1091-4450-46e7-b132-1dd227a60210 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:29.142308 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f1091-4450-46e7-b132-1dd227a60210 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:29.142975 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2f1091-4450-46e7-b132-1dd227a60210 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:29.144848 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2f1091-4450-46e7-b132-1dd227a60210 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:29.207405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1313/2608] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:40:29 2022] POST /identity/v3/projects => generated 379 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:40:29.211454 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14817ea-1b8d-460a-beb7-b57d747c63ac admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:29.264904 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1296/2609] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:40:29 2022] POST /identity/v3/projects => generated 387 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:40:29.269154 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f1091-4450-46e7-b132-1dd227a60210 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:29.274657 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14817ea-1b8d-460a-beb7-b57d747c63ac admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:29.332481 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f1091-4450-46e7-b132-1dd227a60210 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:29.333531 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94758c-5c30-4bd4-97f7-f47bbec0ad90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:29.334141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94758c-5c30-4bd4-97f7-f47bbec0ad90 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:29.334440 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94758c-5c30-4bd4-97f7-f47bbec0ad90 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:29.334719 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94758c-5c30-4bd4-97f7-f47bbec0ad90 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:29.335345 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef94758c-5c30-4bd4-97f7-f47bbec0ad90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:29.337259 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef94758c-5c30-4bd4-97f7-f47bbec0ad90 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:29.337369 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e855386-7d3c-4cd6-b69e-8d37e4bf2ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:29.337968 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e855386-7d3c-4cd6-b69e-8d37e4bf2ef5 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:29.338298 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e855386-7d3c-4cd6-b69e-8d37e4bf2ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:29.338603 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e855386-7d3c-4cd6-b69e-8d37e4bf2ef5 admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:29.339236 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e855386-7d3c-4cd6-b69e-8d37e4bf2ef5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:29.341295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e855386-7d3c-4cd6-b69e-8d37e4bf2ef5 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:29.435083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1314/2610] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:40:29 2022] POST /identity/v3/users => generated 290 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:40:29.440277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94758c-5c30-4bd4-97f7-f47bbec0ad90 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:29.450718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94758c-5c30-4bd4-97f7-f47bbec0ad90 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:29.455957 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc3cf59-e002-4e78-9212-388dc3066be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:29.460626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3cf59-e002-4e78-9212-388dc3066be0 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:29.460626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3cf59-e002-4e78-9212-388dc3066be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:29.460626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3cf59-e002-4e78-9212-388dc3066be0 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:29.460626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc3cf59-e002-4e78-9212-388dc3066be0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:29.460925 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc3cf59-e002-4e78-9212-388dc3066be0 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:29.464008 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1297/2611] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:40:29 2022] POST /identity/v3/users => generated 294 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:40:29.468132 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e855386-7d3c-4cd6-b69e-8d37e4bf2ef5 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:29.471728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1315/2612] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:40:29.476370 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc3cf59-e002-4e78-9212-388dc3066be0 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:29.478517 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e855386-7d3c-4cd6-b69e-8d37e4bf2ef5 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:29.483235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91dcbff-22f8-4bda-8409-81eaf7953dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:29.483867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91dcbff-22f8-4bda-8409-81eaf7953dad admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:29.484143 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91dcbff-22f8-4bda-8409-81eaf7953dad admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:29.484412 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91dcbff-22f8-4bda-8409-81eaf7953dad admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:29.485922 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91dcbff-22f8-4bda-8409-81eaf7953dad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:29.487541 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91dcbff-22f8-4bda-8409-81eaf7953dad admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:29.487652 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc3cf59-e002-4e78-9212-388dc3066be0 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:29.492263 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7858ac6-72dd-4c43-8873-2aa9ad970973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:29.493265 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7858ac6-72dd-4c43-8873-2aa9ad970973 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:29.493265 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7858ac6-72dd-4c43-8873-2aa9ad970973 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:29.493490 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7858ac6-72dd-4c43-8873-2aa9ad970973 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:29.494017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7858ac6-72dd-4c43-8873-2aa9ad970973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:29.495624 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7858ac6-72dd-4c43-8873-2aa9ad970973 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:29.498168 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1298/2613] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:40:29.503248 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91dcbff-22f8-4bda-8409-81eaf7953dad admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:29.512282 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1316/2614] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:40:29.513407 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7858ac6-72dd-4c43-8873-2aa9ad970973 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:29.513527 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91dcbff-22f8-4bda-8409-81eaf7953dad admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:29.518969 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95183df4-8f8f-4f9a-a785-fa2555d16cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:29.519570 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95183df4-8f8f-4f9a-a785-fa2555d16cb7 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:29.519850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95183df4-8f8f-4f9a-a785-fa2555d16cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:29.520121 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95183df4-8f8f-4f9a-a785-fa2555d16cb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:29.520761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95183df4-8f8f-4f9a-a785-fa2555d16cb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:29.522334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95183df4-8f8f-4f9a-a785-fa2555d16cb7 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:29.524050 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7858ac6-72dd-4c43-8873-2aa9ad970973 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:29.528815 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f7b78-2b26-43c7-aa13-ef52b9c43368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:29.529465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f7b78-2b26-43c7-aa13-ef52b9c43368 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:29.529758 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f7b78-2b26-43c7-aa13-ef52b9c43368 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:29.530039 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f7b78-2b26-43c7-aa13-ef52b9c43368 admin admin] PATH_INFO: `/v3/projects/b1c83a9f51634160a0e022671ae2ff5f/users/d750e0aa6c534ad9a76badb8933ba999/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:29.534440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1299/2615] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:40:29.538685 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95183df4-8f8f-4f9a-a785-fa2555d16cb7 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:29.545435 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f7b78-2b26-43c7-aa13-ef52b9c43368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1c83a9f51634160a0e022671ae2ff5f, user_id=d750e0aa6c534ad9a76badb8933ba999, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:29.548846 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f7b78-2b26-43c7-aa13-ef52b9c43368 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:29.549110 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95183df4-8f8f-4f9a-a785-fa2555d16cb7 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:29.553907 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb9fef2-f606-42d1-ab2c-61858aac3932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:29.554597 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9fef2-f606-42d1-ab2c-61858aac3932 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:29.554900 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9fef2-f606-42d1-ab2c-61858aac3932 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:29.555156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9fef2-f606-42d1-ab2c-61858aac3932 admin admin] PATH_INFO: `/v3/projects/d1b506e3229846f79340555bb2af11e0/users/55d158f3580e4087947ee2ee81c60ecb/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:29.570485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb9fef2-f606-42d1-ab2c-61858aac3932 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1b506e3229846f79340555bb2af11e0, user_id=55d158f3580e4087947ee2ee81c60ecb, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:29.573126 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb9fef2-f606-42d1-ab2c-61858aac3932 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:29.623657 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1317/2616] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:40:29 2022] PUT /identity/v3/projects/b1c83a9f51634160a0e022671ae2ff5f/users/d750e0aa6c534ad9a76badb8933ba999/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:29.636370 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e69b8f-5242-4a3f-997e-1cce63b9409f None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:29.636676 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e69b8f-5242-4a3f-997e-1cce63b9409f None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:29.636949 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e69b8f-5242-4a3f-997e-1cce63b9409f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:29.647748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:29.654779 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1300/2617] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:40:29 2022] PUT /identity/v3/projects/d1b506e3229846f79340555bb2af11e0/users/55d158f3580e4087947ee2ee81c60ecb/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:40:29.665720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988a3b3-9b59-474a-8ac5-8b4c3818dca3 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:29.666008 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988a3b3-9b59-474a-8ac5-8b4c3818dca3 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:29.666269 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988a3b3-9b59-474a-8ac5-8b4c3818dca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:29.668345 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-80e69b8f-5242-4a3f-997e-1cce63b9409f None None] MFA Rules not processed for user `d750e0aa6c534ad9a76badb8933ba999`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:29.669528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-80e69b8f-5242-4a3f-997e-1cce63b9409f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:29.694938 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-a988a3b3-9b59-474a-8ac5-8b4c3818dca3 None None] MFA Rules not processed for user `55d158f3580e4087947ee2ee81c60ecb`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:29.696221 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a988a3b3-9b59-474a-8ac5-8b4c3818dca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:29.727882 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:29.738592 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1318/2618] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:40:29.743090 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e69b8f-5242-4a3f-997e-1cce63b9409f None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:29.761061 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1301/2619] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:40:29.767936 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a988a3b3-9b59-474a-8ac5-8b4c3818dca3 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:29.807178 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e69b8f-5242-4a3f-997e-1cce63b9409f None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:29.812792 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3160184-2ee4-4b12-b93b-0b992848d0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:29.813453 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3160184-2ee4-4b12-b93b-0b992848d0c5 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:29.813771 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3160184-2ee4-4b12-b93b-0b992848d0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:29.814063 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3160184-2ee4-4b12-b93b-0b992848d0c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:29.814727 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3160184-2ee4-4b12-b93b-0b992848d0c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:29.816671 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3160184-2ee4-4b12-b93b-0b992848d0c5 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:29.830045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a988a3b3-9b59-474a-8ac5-8b4c3818dca3 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:29.836298 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2255b-3fbe-4d2a-af1e-76ba1c7c0f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:29.836909 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2255b-3fbe-4d2a-af1e-76ba1c7c0f13 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:29.839251 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2255b-3fbe-4d2a-af1e-76ba1c7c0f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:29.839251 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2255b-3fbe-4d2a-af1e-76ba1c7c0f13 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:29.839251 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b2255b-3fbe-4d2a-af1e-76ba1c7c0f13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:29.839997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b2255b-3fbe-4d2a-af1e-76ba1c7c0f13 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:29.893046 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1319/2620] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:40:29 2022] POST /identity/v3/projects => generated 379 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:40:29.896909 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3160184-2ee4-4b12-b93b-0b992848d0c5 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:29.950100 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:29.954421 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1302/2621] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:40:29 2022] POST /identity/v3/projects => generated 387 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:40:29.955419 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2255b-3fbe-4d2a-af1e-76ba1c7c0f13 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:29.961668 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3160184-2ee4-4b12-b93b-0b992848d0c5 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:30.018415 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff89e44-1d9c-48ca-9f90-7891236df5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:30.018674 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2255b-3fbe-4d2a-af1e-76ba1c7c0f13 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:30.019040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff89e44-1d9c-48ca-9f90-7891236df5f6 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:30.019312 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff89e44-1d9c-48ca-9f90-7891236df5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:30.019574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff89e44-1d9c-48ca-9f90-7891236df5f6 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:30.020258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff89e44-1d9c-48ca-9f90-7891236df5f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:30.022126 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff89e44-1d9c-48ca-9f90-7891236df5f6 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:30.023208 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0aef4-cc6d-4240-9018-8d219c35ce91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:30.023807 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0aef4-cc6d-4240-9018-8d219c35ce91 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:30.024078 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0aef4-cc6d-4240-9018-8d219c35ce91 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:30.024342 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0aef4-cc6d-4240-9018-8d219c35ce91 admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:30.024957 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e0aef4-cc6d-4240-9018-8d219c35ce91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:30.026865 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e0aef4-cc6d-4240-9018-8d219c35ce91 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:30.036576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:30.109768 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1320/2622] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:40:29 2022] POST /identity/v3/users => generated 290 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:40:30.114133 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff89e44-1d9c-48ca-9f90-7891236df5f6 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:30.124082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff89e44-1d9c-48ca-9f90-7891236df5f6 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:30.129280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25c56e-e713-4216-b401-5d82751c4af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:30.129857 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25c56e-e713-4216-b401-5d82751c4af9 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:30.130129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25c56e-e713-4216-b401-5d82751c4af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:30.130481 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25c56e-e713-4216-b401-5d82751c4af9 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:30.131783 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee25c56e-e713-4216-b401-5d82751c4af9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:30.133381 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee25c56e-e713-4216-b401-5d82751c4af9 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:30.144342 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1321/2623] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:40:30.148060 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25c56e-e713-4216-b401-5d82751c4af9 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:30.151127 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1303/2624] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:40:29 2022] POST /identity/v3/users => generated 300 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:40:30.158939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25c56e-e713-4216-b401-5d82751c4af9 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:30.162524 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0aef4-cc6d-4240-9018-8d219c35ce91 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:30.163863 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5c090-1af8-42eb-81f6-c4c371f46eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:30.164458 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5c090-1af8-42eb-81f6-c4c371f46eff admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:30.164788 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5c090-1af8-42eb-81f6-c4c371f46eff admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:30.165069 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5c090-1af8-42eb-81f6-c4c371f46eff admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:30.165711 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5c090-1af8-42eb-81f6-c4c371f46eff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:30.167360 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5c090-1af8-42eb-81f6-c4c371f46eff admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:30.172846 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0aef4-cc6d-4240-9018-8d219c35ce91 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:30.177860 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11205ff0-c6d5-433b-939c-718d8840859b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:30.178493 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11205ff0-c6d5-433b-939c-718d8840859b admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:30.178807 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11205ff0-c6d5-433b-939c-718d8840859b admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:30.179021 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1322/2625] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:40:30.179149 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11205ff0-c6d5-433b-939c-718d8840859b admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:30.183726 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11205ff0-c6d5-433b-939c-718d8840859b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:30.183726 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11205ff0-c6d5-433b-939c-718d8840859b admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:30.186668 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5c090-1af8-42eb-81f6-c4c371f46eff admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:30.195960 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1304/2626] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:40:30.196929 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5c090-1af8-42eb-81f6-c4c371f46eff admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:30.199929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11205ff0-c6d5-433b-939c-718d8840859b admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:30.201887 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db9a74-214b-4d37-bf98-27b50ba0ba4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:30.202569 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db9a74-214b-4d37-bf98-27b50ba0ba4f admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:30.202850 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db9a74-214b-4d37-bf98-27b50ba0ba4f admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:30.203124 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db9a74-214b-4d37-bf98-27b50ba0ba4f admin admin] PATH_INFO: `/v3/projects/3ce5829f19a242c680eb1ecb6f06f3fa/users/2da2deba507f476781f80477f0714666/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:30.210359 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11205ff0-c6d5-433b-939c-718d8840859b admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:30.214918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b42260-1f6f-4ad0-be33-4ea1112dd782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:30.215585 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b42260-1f6f-4ad0-be33-4ea1112dd782 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:30.215997 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b42260-1f6f-4ad0-be33-4ea1112dd782 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:30.216258 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b42260-1f6f-4ad0-be33-4ea1112dd782 admin admin] PATH_INFO: `/v3/projects/a3633e6305744c30837dbb85443ddc74/users/dabca430c8344715ba7840c24fb75541/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:30.220016 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db9a74-214b-4d37-bf98-27b50ba0ba4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ce5829f19a242c680eb1ecb6f06f3fa, user_id=2da2deba507f476781f80477f0714666, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:30.222751 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db9a74-214b-4d37-bf98-27b50ba0ba4f admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:30.232298 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b42260-1f6f-4ad0-be33-4ea1112dd782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3633e6305744c30837dbb85443ddc74, user_id=dabca430c8344715ba7840c24fb75541, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:30.234944 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b42260-1f6f-4ad0-be33-4ea1112dd782 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:30.338717 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1323/2627] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:40:30 2022] PUT /identity/v3/projects/3ce5829f19a242c680eb1ecb6f06f3fa/users/2da2deba507f476781f80477f0714666/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:40:30.349208 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad664b5-011d-4d88-9241-f4b169880f4a None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:30.349498 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad664b5-011d-4d88-9241-f4b169880f4a None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:30.349769 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad664b5-011d-4d88-9241-f4b169880f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:30.378935 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-fad664b5-011d-4d88-9241-f4b169880f4a None None] MFA Rules not processed for user `2da2deba507f476781f80477f0714666`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:30.382306 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1305/2628] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:40:30 2022] PUT /identity/v3/projects/a3633e6305744c30837dbb85443ddc74/users/dabca430c8344715ba7840c24fb75541/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:30.382466 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-fad664b5-011d-4d88-9241-f4b169880f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:30.385037 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b42260-1f6f-4ad0-be33-4ea1112dd782 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:30.398811 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:30.454949 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1324/2629] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:40:30.460590 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad664b5-011d-4d88-9241-f4b169880f4a None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:30.469666 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b42260-1f6f-4ad0-be33-4ea1112dd782 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:30.474119 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad664b5-011d-4d88-9241-f4b169880f4a None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:30.475199 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9e0ce-f2ca-4a3c-b348-3283507110dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:30.475854 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9e0ce-f2ca-4a3c-b348-3283507110dc admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:30.476129 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9e0ce-f2ca-4a3c-b348-3283507110dc admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:30.476393 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9e0ce-f2ca-4a3c-b348-3283507110dc admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:30.477321 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a9e0ce-f2ca-4a3c-b348-3283507110dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:30.477513 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f07f8e-a8b4-4a02-b158-5bd21377767b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:30.478206 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f07f8e-a8b4-4a02-b158-5bd21377767b admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:30.478421 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f07f8e-a8b4-4a02-b158-5bd21377767b admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:30.478696 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f07f8e-a8b4-4a02-b158-5bd21377767b admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:30.478947 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a9e0ce-f2ca-4a3c-b348-3283507110dc admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:30.479329 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f07f8e-a8b4-4a02-b158-5bd21377767b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:30.481678 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f07f8e-a8b4-4a02-b158-5bd21377767b admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:30.490036 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1306/2630] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:40:30.494321 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9e0ce-f2ca-4a3c-b348-3283507110dc admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:30.506072 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9e0ce-f2ca-4a3c-b348-3283507110dc admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:30.511204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503b6b9-c720-4bb0-9327-dea409be0b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:30.514507 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503b6b9-c720-4bb0-9327-dea409be0b86 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:30.514507 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503b6b9-c720-4bb0-9327-dea409be0b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:30.514507 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503b6b9-c720-4bb0-9327-dea409be0b86 admin admin] PATH_INFO: `/v3/projects/a3633e6305744c30837dbb85443ddc74/users/dabca430c8344715ba7840c24fb75541/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:30.514507 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503b6b9-c720-4bb0-9327-dea409be0b86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3633e6305744c30837dbb85443ddc74, user_id=dabca430c8344715ba7840c24fb75541, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:30.518355 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503b6b9-c720-4bb0-9327-dea409be0b86 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:30.529591 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1307/2631] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:40:30 2022] PUT /identity/v3/projects/a3633e6305744c30837dbb85443ddc74/users/dabca430c8344715ba7840c24fb75541/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:40:30.540925 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea817f1-f86e-489d-bde5-3cca3a577340 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:30.541208 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea817f1-f86e-489d-bde5-3cca3a577340 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:30.541485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea817f1-f86e-489d-bde5-3cca3a577340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:30.572169 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-7ea817f1-f86e-489d-bde5-3cca3a577340 None None] MFA Rules not processed for user `dabca430c8344715ba7840c24fb75541`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:30.573386 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-7ea817f1-f86e-489d-bde5-3cca3a577340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:30.625305 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1325/2632] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:40:30 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:40:30.630269 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f07f8e-a8b4-4a02-b158-5bd21377767b admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:30.649304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1308/2633] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:30 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:40:30.661126 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea817f1-f86e-489d-bde5-3cca3a577340 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:30.662708 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-7ea817f1-f86e-489d-bde5-3cca3a577340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:30.692567 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:30.693118 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f07f8e-a8b4-4a02-b158-5bd21377767b admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:30.697895 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf622311-3b2c-477e-b9b9-639781a578a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:30.698532 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf622311-3b2c-477e-b9b9-639781a578a8 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:30.698812 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf622311-3b2c-477e-b9b9-639781a578a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:30.699083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf622311-3b2c-477e-b9b9-639781a578a8 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:30.699719 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf622311-3b2c-477e-b9b9-639781a578a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:30.701649 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf622311-3b2c-477e-b9b9-639781a578a8 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:30.724611 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea817f1-f86e-489d-bde5-3cca3a577340 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:30.775602 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1ed3a-5736-416b-a66b-1d85a4f7696d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:30.776237 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1ed3a-5736-416b-a66b-1d85a4f7696d service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:30.776514 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1ed3a-5736-416b-a66b-1d85a4f7696d service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:30.776781 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1ed3a-5736-416b-a66b-1d85a4f7696d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:30.807224 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1326/2634] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:40:30 2022] POST /identity/v3/users => generated 295 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:40:30.811077 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf622311-3b2c-477e-b9b9-639781a578a8 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:30.821742 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf622311-3b2c-477e-b9b9-639781a578a8 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:30.827306 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e120c06f-1bd1-4104-96ee-95b5acb89f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:30.827906 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e120c06f-1bd1-4104-96ee-95b5acb89f47 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:30.828184 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e120c06f-1bd1-4104-96ee-95b5acb89f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:30.829015 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e120c06f-1bd1-4104-96ee-95b5acb89f47 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:30.829705 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e120c06f-1bd1-4104-96ee-95b5acb89f47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:30.831367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e120c06f-1bd1-4104-96ee-95b5acb89f47 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:30.842236 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1327/2635] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:40:30.846946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e120c06f-1bd1-4104-96ee-95b5acb89f47 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:30.857533 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e120c06f-1bd1-4104-96ee-95b5acb89f47 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:30.862816 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb1d12-108a-42b6-8b76-18b1fbab806e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:30.863475 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb1d12-108a-42b6-8b76-18b1fbab806e admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:30.863753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb1d12-108a-42b6-8b76-18b1fbab806e admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:30.864022 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb1d12-108a-42b6-8b76-18b1fbab806e admin admin] PATH_INFO: `/v3/projects/7ca0dfd1456e4a2590f3a639b0e0ced3/users/d598bc3cd9f342ff8a321bffdb1359a0/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:30.865917 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1ed3a-5736-416b-a66b-1d85a4f7696d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:30.867393 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1ed3a-5736-416b-a66b-1d85a4f7696d service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:30.879213 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1309/2636] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:40:30 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:40:30.880665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abb1d12-108a-42b6-8b76-18b1fbab806e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca0dfd1456e4a2590f3a639b0e0ced3, user_id=d598bc3cd9f342ff8a321bffdb1359a0, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:30.883376 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abb1d12-108a-42b6-8b76-18b1fbab806e admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:30.973882 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1328/2637] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:40:30 2022] PUT /identity/v3/projects/7ca0dfd1456e4a2590f3a639b0e0ced3/users/d598bc3cd9f342ff8a321bffdb1359a0/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:40:30.977917 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1ed3a-5736-416b-a66b-1d85a4f7696d service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:30.991026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1ed3a-5736-416b-a66b-1d85a4f7696d service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:31.060658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e662fd-732e-47e2-afb1-00a697ab56f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:31.061485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e662fd-732e-47e2-afb1-00a697ab56f6 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:31.061845 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e662fd-732e-47e2-afb1-00a697ab56f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:31.062175 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e662fd-732e-47e2-afb1-00a697ab56f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:31.063532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e662fd-732e-47e2-afb1-00a697ab56f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:31.065216 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e662fd-732e-47e2-afb1-00a697ab56f6 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:31.076048 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1310/2638] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:40:31.082472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb1d12-108a-42b6-8b76-18b1fbab806e admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:31.094362 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb1d12-108a-42b6-8b76-18b1fbab806e admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:31.099540 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05750ed-b4c4-4bda-b38a-c6da16290114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:31.100205 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05750ed-b4c4-4bda-b38a-c6da16290114 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:31.107443 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05750ed-b4c4-4bda-b38a-c6da16290114 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:31.107443 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05750ed-b4c4-4bda-b38a-c6da16290114 admin admin] PATH_INFO: `/v3/projects/7ca0dfd1456e4a2590f3a639b0e0ced3/users/d598bc3cd9f342ff8a321bffdb1359a0/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:31.107443 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05750ed-b4c4-4bda-b38a-c6da16290114 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca0dfd1456e4a2590f3a639b0e0ced3, user_id=d598bc3cd9f342ff8a321bffdb1359a0, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:31.107443 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05750ed-b4c4-4bda-b38a-c6da16290114 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:31.117625 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1329/2639] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:40:31 2022] PUT /identity/v3/projects/7ca0dfd1456e4a2590f3a639b0e0ced3/users/d598bc3cd9f342ff8a321bffdb1359a0/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:31.130497 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426a9ce-f44a-4017-aad4-422700c4a96b None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:31.130781 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426a9ce-f44a-4017-aad4-422700c4a96b None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:31.131055 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426a9ce-f44a-4017-aad4-422700c4a96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:31.159613 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-a426a9ce-f44a-4017-aad4-422700c4a96b None None] MFA Rules not processed for user `d598bc3cd9f342ff8a321bffdb1359a0`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:31.160847 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a426a9ce-f44a-4017-aad4-422700c4a96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:31.176201 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:31.237299 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1311/2640] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:31 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:40:31.249052 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05750ed-b4c4-4bda-b38a-c6da16290114 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:31.292817 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:31.307678 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05750ed-b4c4-4bda-b38a-c6da16290114 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:31.311323 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859ea37-e1fa-4900-8462-fd0bdd277767 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:31.311946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859ea37-e1fa-4900-8462-fd0bdd277767 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:31.312265 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859ea37-e1fa-4900-8462-fd0bdd277767 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:31.312537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859ea37-e1fa-4900-8462-fd0bdd277767 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:31.324163 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6859ea37-e1fa-4900-8462-fd0bdd277767 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:31.325904 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6859ea37-e1fa-4900-8462-fd0bdd277767 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:31.336725 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1330/2641] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:40:31 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:40:31.499971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426a9ce-f44a-4017-aad4-422700c4a96b None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:31.502813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a426a9ce-f44a-4017-aad4-422700c4a96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:31.518939 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426a9ce-f44a-4017-aad4-422700c4a96b None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:31.525262 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a9721-cab0-4b68-b957-b925de63387c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:31.525262 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a9721-cab0-4b68-b957-b925de63387c service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:31.525262 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a9721-cab0-4b68-b957-b925de63387c service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:31.525262 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a9721-cab0-4b68-b957-b925de63387c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:31.582994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a9721-cab0-4b68-b957-b925de63387c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:31.584432 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a9721-cab0-4b68-b957-b925de63387c service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:31.593393 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1312/2642] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:40:31 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:40:32.170718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859ea37-e1fa-4900-8462-fd0bdd277767 service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:32.236141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859ea37-e1fa-4900-8462-fd0bdd277767 service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:32.240777 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c25f2-03c2-4548-8445-bb294b771569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:32.241396 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c25f2-03c2-4548-8445-bb294b771569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:32.241674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c25f2-03c2-4548-8445-bb294b771569 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:32.241946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c25f2-03c2-4548-8445-bb294b771569 admin admin] PATH_INFO: `/v3/users/55d158f3580e4087947ee2ee81c60ecb` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:32.243079 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101c25f2-03c2-4548-8445-bb294b771569 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55d158f3580e4087947ee2ee81c60ecb)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:32.245038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101c25f2-03c2-4548-8445-bb294b771569 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:32.420635 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-101c25f2-03c2-4548-8445-bb294b771569 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55d158f3580e4087947ee2ee81c60ecb'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:32.435567 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-101c25f2-03c2-4548-8445-bb294b771569 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55d158f3580e4087947ee2ee81c60ecb'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:32.445080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-101c25f2-03c2-4548-8445-bb294b771569 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55d158f3580e4087947ee2ee81c60ecb'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:32.560202 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-101c25f2-03c2-4548-8445-bb294b771569 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55d158f3580e4087947ee2ee81c60ecb'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:32.560864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-101c25f2-03c2-4548-8445-bb294b771569 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55d158f3580e4087947ee2ee81c60ecb'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:32.567789 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1331/2643] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:40:32 2022] DELETE /identity/v3/users/55d158f3580e4087947ee2ee81c60ecb => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:32.572602 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7fda7-0df3-4080-b850-907f0e847a3d None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:32.573065 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7fda7-0df3-4080-b850-907f0e847a3d None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:32.573150 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7fda7-0df3-4080-b850-907f0e847a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:32.593817 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-0cf7fda7-0df3-4080-b850-907f0e847a3d None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:32.599689 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-0cf7fda7-0df3-4080-b850-907f0e847a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:32.658624 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1313/2644] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:40:32.668517 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c25f2-03c2-4548-8445-bb294b771569 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:32.669935 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-101c25f2-03c2-4548-8445-bb294b771569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:32.731083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c25f2-03c2-4548-8445-bb294b771569 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:32.736525 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24564c-262a-4576-bb56-b6bbd3a2fa96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:32.737298 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24564c-262a-4576-bb56-b6bbd3a2fa96 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:32.737579 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24564c-262a-4576-bb56-b6bbd3a2fa96 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:32.737858 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24564c-262a-4576-bb56-b6bbd3a2fa96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:32.749251 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f24564c-262a-4576-bb56-b6bbd3a2fa96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:32.751075 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f24564c-262a-4576-bb56-b6bbd3a2fa96 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:32.761044 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1332/2645] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:40:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:40:33.168158 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7fda7-0df3-4080-b850-907f0e847a3d None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:33.169848 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-0cf7fda7-0df3-4080-b850-907f0e847a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:33.182882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7fda7-0df3-4080-b850-907f0e847a3d None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:33.188125 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1bbfab-3cda-48b7-b2c7-e5f63371eb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:33.188790 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bbfab-3cda-48b7-b2c7-e5f63371eb10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:33.189068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bbfab-3cda-48b7-b2c7-e5f63371eb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:33.189327 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bbfab-3cda-48b7-b2c7-e5f63371eb10 admin admin] PATH_INFO: `/v3/projects/d1b506e3229846f79340555bb2af11e0` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:33.190608 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1bbfab-3cda-48b7-b2c7-e5f63371eb10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1b506e3229846f79340555bb2af11e0)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:33.192520 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1bbfab-3cda-48b7-b2c7-e5f63371eb10 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:33.295349 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:33.298367 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5b1bbfab-3cda-48b7-b2c7-e5f63371eb10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1b506e3229846f79340555bb2af11e0', 'request_id': 'req-5b1bbfab-3cda-48b7-b2c7-e5f63371eb10'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:33.304931 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5b1bbfab-3cda-48b7-b2c7-e5f63371eb10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1b506e3229846f79340555bb2af11e0', 'request_id': 'req-5b1bbfab-3cda-48b7-b2c7-e5f63371eb10'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:33.310716 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5b1bbfab-3cda-48b7-b2c7-e5f63371eb10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1b506e3229846f79340555bb2af11e0', 'request_id': 'req-5b1bbfab-3cda-48b7-b2c7-e5f63371eb10'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:33.429746 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5b1bbfab-3cda-48b7-b2c7-e5f63371eb10 admin admin] The token cache is being invalidate because project d1b506e3229846f79340555bb2af11e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:40:33.430124 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5b1bbfab-3cda-48b7-b2c7-e5f63371eb10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:33.430994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5b1bbfab-3cda-48b7-b2c7-e5f63371eb10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:33.432636 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1314/2646] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:40:33 2022] DELETE /identity/v3/projects/d1b506e3229846f79340555bb2af11e0 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:33.438341 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24564c-262a-4576-bb56-b6bbd3a2fa96 service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:33.445563 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:33.452349 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24564c-262a-4576-bb56-b6bbd3a2fa96 service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:33.510976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b917462-9511-43ed-a251-cc15e133d1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:33.511653 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b917462-9511-43ed-a251-cc15e133d1bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:33.511933 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b917462-9511-43ed-a251-cc15e133d1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:33.512203 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b917462-9511-43ed-a251-cc15e133d1bc admin admin] PATH_INFO: `/v3/users/dabca430c8344715ba7840c24fb75541` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:33.513373 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b917462-9511-43ed-a251-cc15e133d1bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=dabca430c8344715ba7840c24fb75541)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:33.515313 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b917462-9511-43ed-a251-cc15e133d1bc admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:33.728803 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-4b917462-9511-43ed-a251-cc15e133d1bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dabca430c8344715ba7840c24fb75541'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:33.741512 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-4b917462-9511-43ed-a251-cc15e133d1bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dabca430c8344715ba7840c24fb75541'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:33.748421 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-4b917462-9511-43ed-a251-cc15e133d1bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dabca430c8344715ba7840c24fb75541'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:33.822973 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-4b917462-9511-43ed-a251-cc15e133d1bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dabca430c8344715ba7840c24fb75541'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:33.823696 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-4b917462-9511-43ed-a251-cc15e133d1bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dabca430c8344715ba7840c24fb75541'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:33.830722 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1333/2647] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:40:33 2022] DELETE /identity/v3/users/dabca430c8344715ba7840c24fb75541 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:34.316129 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1bbfab-3cda-48b7-b2c7-e5f63371eb10 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:34.317580 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-5b1bbfab-3cda-48b7-b2c7-e5f63371eb10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:34.332049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1bbfab-3cda-48b7-b2c7-e5f63371eb10 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:34.392759 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52984a4f-aadd-44d0-9902-9a307f1f8ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:34.393385 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52984a4f-aadd-44d0-9902-9a307f1f8ac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:34.393802 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52984a4f-aadd-44d0-9902-9a307f1f8ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:34.394080 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52984a4f-aadd-44d0-9902-9a307f1f8ac8 admin admin] PATH_INFO: `/v3/projects/a3633e6305744c30837dbb85443ddc74` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:34.395254 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52984a4f-aadd-44d0-9902-9a307f1f8ac8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3633e6305744c30837dbb85443ddc74)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:34.397154 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52984a4f-aadd-44d0-9902-9a307f1f8ac8 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:34.505542 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-52984a4f-aadd-44d0-9902-9a307f1f8ac8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3633e6305744c30837dbb85443ddc74', 'request_id': 'req-52984a4f-aadd-44d0-9902-9a307f1f8ac8'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:34.512297 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-52984a4f-aadd-44d0-9902-9a307f1f8ac8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3633e6305744c30837dbb85443ddc74', 'request_id': 'req-52984a4f-aadd-44d0-9902-9a307f1f8ac8'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:34.518673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-52984a4f-aadd-44d0-9902-9a307f1f8ac8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3633e6305744c30837dbb85443ddc74', 'request_id': 'req-52984a4f-aadd-44d0-9902-9a307f1f8ac8'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:34.605641 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-52984a4f-aadd-44d0-9902-9a307f1f8ac8 admin admin] The token cache is being invalidate because project a3633e6305744c30837dbb85443ddc74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:40:34.606035 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-52984a4f-aadd-44d0-9902-9a307f1f8ac8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:34.606788 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-52984a4f-aadd-44d0-9902-9a307f1f8ac8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:34.608420 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1315/2648] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:40:34 2022] DELETE /identity/v3/projects/a3633e6305744c30837dbb85443ddc74 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:34.617520 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b43621-9dd6-41f8-bbd1-27efaa0981c7 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:34.618028 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b43621-9dd6-41f8-bbd1-27efaa0981c7 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:34.618104 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b43621-9dd6-41f8-bbd1-27efaa0981c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:34.640370 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-05b43621-9dd6-41f8-bbd1-27efaa0981c7 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:34.642244 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-05b43621-9dd6-41f8-bbd1-27efaa0981c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:34.719519 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1334/2649] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:40:34.730568 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefd8c2-9692-4c6f-a74c-7aaa39fb59b8 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:34.731001 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefd8c2-9692-4c6f-a74c-7aaa39fb59b8 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:34.731152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefd8c2-9692-4c6f-a74c-7aaa39fb59b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:34.753780 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-0aefd8c2-9692-4c6f-a74c-7aaa39fb59b8 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:34.755331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-0aefd8c2-9692-4c6f-a74c-7aaa39fb59b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:34.763813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1316/2650] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:40:34.769279 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b43621-9dd6-41f8-bbd1-27efaa0981c7 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:34.779959 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b43621-9dd6-41f8-bbd1-27efaa0981c7 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:34.785082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af7bc2-dd07-4b0e-9436-4aaf0853dc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:34.785723 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af7bc2-dd07-4b0e-9436-4aaf0853dc0b admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:34.785981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af7bc2-dd07-4b0e-9436-4aaf0853dc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:34.786250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af7bc2-dd07-4b0e-9436-4aaf0853dc0b admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:34.786943 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af7bc2-dd07-4b0e-9436-4aaf0853dc0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:40:34.787244 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af7bc2-dd07-4b0e-9436-4aaf0853dc0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:34.788943 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af7bc2-dd07-4b0e-9436-4aaf0853dc0b admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:34.805633 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1335/2651] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:40:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:40:34.810635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aefd8c2-9692-4c6f-a74c-7aaa39fb59b8 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:34.821970 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aefd8c2-9692-4c6f-a74c-7aaa39fb59b8 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:34.829000 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e91fe-c236-445a-87a9-5ca4650dcae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:34.829533 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e91fe-c236-445a-87a9-5ca4650dcae4 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:34.829988 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e91fe-c236-445a-87a9-5ca4650dcae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:34.830262 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e91fe-c236-445a-87a9-5ca4650dcae4 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:34.830941 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e91fe-c236-445a-87a9-5ca4650dcae4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:34.833121 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e91fe-c236-445a-87a9-5ca4650dcae4 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:34.912347 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1317/2652] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:40:34 2022] POST /identity/v3/projects => generated 371 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:40:34.916525 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af7bc2-dd07-4b0e-9436-4aaf0853dc0b admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:34.927614 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af7bc2-dd07-4b0e-9436-4aaf0853dc0b admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:34.933824 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:34.991311 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a214960-78f7-454e-b952-55543c7fc5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:34.991910 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a214960-78f7-454e-b952-55543c7fc5ad admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:34.992186 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a214960-78f7-454e-b952-55543c7fc5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:34.992456 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a214960-78f7-454e-b952-55543c7fc5ad admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:34.993258 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a214960-78f7-454e-b952-55543c7fc5ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:34.995205 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a214960-78f7-454e-b952-55543c7fc5ad admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:35.082198 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1336/2653] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:40:34 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:40:35.085190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e91fe-c236-445a-87a9-5ca4650dcae4 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:35.097652 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e91fe-c236-445a-87a9-5ca4650dcae4 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:35.103385 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b768a0d-c89d-4ff5-ae4d-dee76ebcc2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:35.104457 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b768a0d-c89d-4ff5-ae4d-dee76ebcc2fc admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:35.104735 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b768a0d-c89d-4ff5-ae4d-dee76ebcc2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:35.105121 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b768a0d-c89d-4ff5-ae4d-dee76ebcc2fc admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:35.105778 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b768a0d-c89d-4ff5-ae4d-dee76ebcc2fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:35.107768 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b768a0d-c89d-4ff5-ae4d-dee76ebcc2fc admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:35.120549 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1318/2654] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:40:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:40:35.124578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a214960-78f7-454e-b952-55543c7fc5ad admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:35.136084 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a214960-78f7-454e-b952-55543c7fc5ad admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:35.140914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace347c-b962-4e89-82af-d3ab55e818e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:35.141645 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace347c-b962-4e89-82af-d3ab55e818e3 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:35.141925 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace347c-b962-4e89-82af-d3ab55e818e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:35.142257 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace347c-b962-4e89-82af-d3ab55e818e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:35.142956 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ace347c-b962-4e89-82af-d3ab55e818e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:35.144586 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ace347c-b962-4e89-82af-d3ab55e818e3 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:35.156156 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1337/2655] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:40:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:40:35.165311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b768a0d-c89d-4ff5-ae4d-dee76ebcc2fc admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:35.176401 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b768a0d-c89d-4ff5-ae4d-dee76ebcc2fc admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:35.181368 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab2dec-17f8-401a-8221-b0cbd85ce512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:35.182147 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab2dec-17f8-401a-8221-b0cbd85ce512 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:35.185252 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab2dec-17f8-401a-8221-b0cbd85ce512 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:35.185252 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab2dec-17f8-401a-8221-b0cbd85ce512 admin admin] PATH_INFO: `/v3/projects/3d62e1f2f3ca4bc19e8f7787d8157d5b/users/0918a457ebf34927854aec9af83eb9de/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:35.199441 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab2dec-17f8-401a-8221-b0cbd85ce512 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d62e1f2f3ca4bc19e8f7787d8157d5b, user_id=0918a457ebf34927854aec9af83eb9de, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:35.202076 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab2dec-17f8-401a-8221-b0cbd85ce512 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:35.300500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:35.310794 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1319/2656] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:40:35 2022] PUT /identity/v3/projects/3d62e1f2f3ca4bc19e8f7787d8157d5b/users/0918a457ebf34927854aec9af83eb9de/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:35.316176 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadff03-75b9-49c1-a624-b4e01d72935e None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:35.316465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadff03-75b9-49c1-a624-b4e01d72935e None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:35.316736 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadff03-75b9-49c1-a624-b4e01d72935e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:35.338283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-2eadff03-75b9-49c1-a624-b4e01d72935e None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:35.339718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2eadff03-75b9-49c1-a624-b4e01d72935e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:35.407251 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1338/2657] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:40:35.417072 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab2dec-17f8-401a-8221-b0cbd85ce512 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:35.418930 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-0dab2dec-17f8-401a-8221-b0cbd85ce512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:35.483612 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab2dec-17f8-401a-8221-b0cbd85ce512 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:35.488192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574021b3-e17b-4ce6-a417-9c4df9fcaefb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:35.489758 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574021b3-e17b-4ce6-a417-9c4df9fcaefb service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:35.489758 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574021b3-e17b-4ce6-a417-9c4df9fcaefb service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:35.489758 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574021b3-e17b-4ce6-a417-9c4df9fcaefb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:35.499180 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574021b3-e17b-4ce6-a417-9c4df9fcaefb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:35.501700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574021b3-e17b-4ce6-a417-9c4df9fcaefb service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:35.513419 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1320/2658] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:40:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:40:36.753483 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eadff03-75b9-49c1-a624-b4e01d72935e None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:36.754962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2eadff03-75b9-49c1-a624-b4e01d72935e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:36.767335 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eadff03-75b9-49c1-a624-b4e01d72935e None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:36.772202 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7533fc-cab9-4be1-b744-1817b0961d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:36.780432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7533fc-cab9-4be1-b744-1817b0961d53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:36.780432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7533fc-cab9-4be1-b744-1817b0961d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:36.780432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7533fc-cab9-4be1-b744-1817b0961d53 admin admin] PATH_INFO: `/v3/users/d750e0aa6c534ad9a76badb8933ba999` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:36.780432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7533fc-cab9-4be1-b744-1817b0961d53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d750e0aa6c534ad9a76badb8933ba999)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:36.780432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7533fc-cab9-4be1-b744-1817b0961d53 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:36.959805 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-5c7533fc-cab9-4be1-b744-1817b0961d53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd750e0aa6c534ad9a76badb8933ba999'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:36.974311 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-5c7533fc-cab9-4be1-b744-1817b0961d53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd750e0aa6c534ad9a76badb8933ba999'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:36.982447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-5c7533fc-cab9-4be1-b744-1817b0961d53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd750e0aa6c534ad9a76badb8933ba999'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:37.039918 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-5c7533fc-cab9-4be1-b744-1817b0961d53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd750e0aa6c534ad9a76badb8933ba999'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:37.040839 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-5c7533fc-cab9-4be1-b744-1817b0961d53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd750e0aa6c534ad9a76badb8933ba999'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:37.043724 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:37.049328 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1339/2659] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:40:36 2022] DELETE /identity/v3/users/d750e0aa6c534ad9a76badb8933ba999 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:37.053908 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94606b77-8236-4a87-8d65-c923b0eec318 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:37.054486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94606b77-8236-4a87-8d65-c923b0eec318 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:37.054486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94606b77-8236-4a87-8d65-c923b0eec318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:37.075354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-94606b77-8236-4a87-8d65-c923b0eec318 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:37.076885 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-94606b77-8236-4a87-8d65-c923b0eec318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:37.138418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0231347-3171-4594-b5a3-00c0ff6faca2 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:37.138710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0231347-3171-4594-b5a3-00c0ff6faca2 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:37.138987 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0231347-3171-4594-b5a3-00c0ff6faca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:37.176824 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1321/2660] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:40:37.181440 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-c0231347-3171-4594-b5a3-00c0ff6faca2 None None] MFA Rules not processed for user `0918a457ebf34927854aec9af83eb9de`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:37.183364 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-c0231347-3171-4594-b5a3-00c0ff6faca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:37.184123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94606b77-8236-4a87-8d65-c923b0eec318 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:37.185645 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-94606b77-8236-4a87-8d65-c923b0eec318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:37.262879 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94606b77-8236-4a87-8d65-c923b0eec318 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:37.266751 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1340/2661] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:40:37.267691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e92c1-cb96-4b62-b1ca-68f5378810fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:37.268346 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e92c1-cb96-4b62-b1ca-68f5378810fa service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:37.268622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e92c1-cb96-4b62-b1ca-68f5378810fa service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:37.268892 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e92c1-cb96-4b62-b1ca-68f5378810fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:37.271477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0231347-3171-4594-b5a3-00c0ff6faca2 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:37.272898 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-c0231347-3171-4594-b5a3-00c0ff6faca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:37.279152 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e92c1-cb96-4b62-b1ca-68f5378810fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:37.280612 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e92c1-cb96-4b62-b1ca-68f5378810fa service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:37.287607 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0231347-3171-4594-b5a3-00c0ff6faca2 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:37.291882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1322/2662] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:40:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:40:37.293826 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf5531-3213-482c-9caa-024f674edf16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:37.294493 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf5531-3213-482c-9caa-024f674edf16 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:37.294771 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf5531-3213-482c-9caa-024f674edf16 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:37.295034 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf5531-3213-482c-9caa-024f674edf16 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:37.295674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf5531-3213-482c-9caa-024f674edf16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:37.297954 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf5531-3213-482c-9caa-024f674edf16 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:37.401779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1341/2663] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:40:37 2022] POST /identity/v3/projects => generated 371 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:40:37.405817 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e92c1-cb96-4b62-b1ca-68f5378810fa service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:37.415681 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e92c1-cb96-4b62-b1ca-68f5378810fa service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:37.463192 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=100 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:37.487559 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517a301-f835-4086-a05d-a213e041a4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:37.488334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517a301-f835-4086-a05d-a213e041a4aa admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:37.488608 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517a301-f835-4086-a05d-a213e041a4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:37.489002 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517a301-f835-4086-a05d-a213e041a4aa admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:37.489652 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e517a301-f835-4086-a05d-a213e041a4aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:37.491605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e517a301-f835-4086-a05d-a213e041a4aa admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:37.583268 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1323/2664] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:40:37 2022] POST /identity/v3/users => generated 292 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:40:37.587378 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf5531-3213-482c-9caa-024f674edf16 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:37.598616 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf5531-3213-482c-9caa-024f674edf16 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:37.603753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4197de7-dfeb-41e1-8402-9ad780f2e18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:37.604441 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4197de7-dfeb-41e1-8402-9ad780f2e18d admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:37.604742 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4197de7-dfeb-41e1-8402-9ad780f2e18d admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:37.605009 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4197de7-dfeb-41e1-8402-9ad780f2e18d admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:37.605647 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4197de7-dfeb-41e1-8402-9ad780f2e18d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:37.607267 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4197de7-dfeb-41e1-8402-9ad780f2e18d admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:37.618458 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1342/2665] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:40:37.622752 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517a301-f835-4086-a05d-a213e041a4aa admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:37.634850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517a301-f835-4086-a05d-a213e041a4aa admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:37.639553 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5a5e8-0073-4bdc-975e-ba4d7f68b2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:37.640257 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5a5e8-0073-4bdc-975e-ba4d7f68b2dd admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:37.640674 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5a5e8-0073-4bdc-975e-ba4d7f68b2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:37.640947 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5a5e8-0073-4bdc-975e-ba4d7f68b2dd admin admin] PATH_INFO: `/v3/projects/1f43f730f3e041ada9987e59d235a582/users/4cddd65cf179486abe179b2382b0d5b4/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:37.656216 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a5a5e8-0073-4bdc-975e-ba4d7f68b2dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f43f730f3e041ada9987e59d235a582, user_id=4cddd65cf179486abe179b2382b0d5b4, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:37.659020 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a5a5e8-0073-4bdc-975e-ba4d7f68b2dd admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:37.755920 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4197de7-dfeb-41e1-8402-9ad780f2e18d admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:37.755920 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-b4197de7-dfeb-41e1-8402-9ad780f2e18d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:37.768574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4197de7-dfeb-41e1-8402-9ad780f2e18d admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:37.773781 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efe48a-904c-44c3-9e1f-beb94699ccbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:37.774441 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efe48a-904c-44c3-9e1f-beb94699ccbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:37.774712 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efe48a-904c-44c3-9e1f-beb94699ccbc admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:37.775053 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efe48a-904c-44c3-9e1f-beb94699ccbc admin admin] PATH_INFO: `/v3/projects/b1c83a9f51634160a0e022671ae2ff5f` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:37.776316 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78efe48a-904c-44c3-9e1f-beb94699ccbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1c83a9f51634160a0e022671ae2ff5f)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:37.778346 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78efe48a-904c-44c3-9e1f-beb94699ccbc admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:37.784676 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1324/2666] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:40:37 2022] PUT /identity/v3/projects/1f43f730f3e041ada9987e59d235a582/users/4cddd65cf179486abe179b2382b0d5b4/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:40:37.789582 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5a5e8-0073-4bdc-975e-ba4d7f68b2dd admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:37.852886 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5a5e8-0073-4bdc-975e-ba4d7f68b2dd admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:37.857851 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62113163-8445-45a2-bc31-482f06e59d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:37.858480 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62113163-8445-45a2-bc31-482f06e59d29 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:37.858756 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62113163-8445-45a2-bc31-482f06e59d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:37.859027 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62113163-8445-45a2-bc31-482f06e59d29 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:37.860381 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62113163-8445-45a2-bc31-482f06e59d29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:37.861987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62113163-8445-45a2-bc31-482f06e59d29 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:37.872056 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1325/2667] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:40:37.876391 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62113163-8445-45a2-bc31-482f06e59d29 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:37.885976 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62113163-8445-45a2-bc31-482f06e59d29 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:37.892210 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639b93d-9192-4113-9838-f53ba38d8022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:37.892858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639b93d-9192-4113-9838-f53ba38d8022 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:37.893172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639b93d-9192-4113-9838-f53ba38d8022 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:37.893452 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639b93d-9192-4113-9838-f53ba38d8022 admin admin] PATH_INFO: `/v3/projects/1f43f730f3e041ada9987e59d235a582/users/4cddd65cf179486abe179b2382b0d5b4/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:37.895539 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f639b93d-9192-4113-9838-f53ba38d8022 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f43f730f3e041ada9987e59d235a582, user_id=4cddd65cf179486abe179b2382b0d5b4, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:37.898155 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f639b93d-9192-4113-9838-f53ba38d8022 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:37.903082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-78efe48a-904c-44c3-9e1f-beb94699ccbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1c83a9f51634160a0e022671ae2ff5f', 'request_id': 'req-78efe48a-904c-44c3-9e1f-beb94699ccbc'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:37.909958 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1326/2668] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:40:37 2022] PUT /identity/v3/projects/1f43f730f3e041ada9987e59d235a582/users/4cddd65cf179486abe179b2382b0d5b4/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:40:37.970996 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-78efe48a-904c-44c3-9e1f-beb94699ccbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1c83a9f51634160a0e022671ae2ff5f', 'request_id': 'req-78efe48a-904c-44c3-9e1f-beb94699ccbc'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:37.977118 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-78efe48a-904c-44c3-9e1f-beb94699ccbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1c83a9f51634160a0e022671ae2ff5f', 'request_id': 'req-78efe48a-904c-44c3-9e1f-beb94699ccbc'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:37.989750 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-78efe48a-904c-44c3-9e1f-beb94699ccbc admin admin] The token cache is being invalidate because project b1c83a9f51634160a0e022671ae2ff5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:40:37.990033 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-78efe48a-904c-44c3-9e1f-beb94699ccbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:37.990741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-78efe48a-904c-44c3-9e1f-beb94699ccbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:37.992260 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1343/2669] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:40:37 2022] DELETE /identity/v3/projects/b1c83a9f51634160a0e022671ae2ff5f => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:40:37.996283 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639b93d-9192-4113-9838-f53ba38d8022 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:37.998131 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-f639b93d-9192-4113-9838-f53ba38d8022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:38.076224 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639b93d-9192-4113-9838-f53ba38d8022 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:38.081178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86c849-0d5c-4c96-b1a3-4df0aaf0725b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:38.082062 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86c849-0d5c-4c96-b1a3-4df0aaf0725b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:38.082331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86c849-0d5c-4c96-b1a3-4df0aaf0725b admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:38.082640 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86c849-0d5c-4c96-b1a3-4df0aaf0725b admin admin] PATH_INFO: `/v3/users/2da2deba507f476781f80477f0714666` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:38.083811 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e86c849-0d5c-4c96-b1a3-4df0aaf0725b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2da2deba507f476781f80477f0714666)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:38.085727 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e86c849-0d5c-4c96-b1a3-4df0aaf0725b admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:38.275865 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:38.285952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-1e86c849-0d5c-4c96-b1a3-4df0aaf0725b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2da2deba507f476781f80477f0714666'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:38.386253 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-1e86c849-0d5c-4c96-b1a3-4df0aaf0725b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2da2deba507f476781f80477f0714666'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:38.388213 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-1e86c849-0d5c-4c96-b1a3-4df0aaf0725b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2da2deba507f476781f80477f0714666'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:38.388844 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-1e86c849-0d5c-4c96-b1a3-4df0aaf0725b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2da2deba507f476781f80477f0714666'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:38.402857 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-1e86c849-0d5c-4c96-b1a3-4df0aaf0725b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2da2deba507f476781f80477f0714666'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:38.417169 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1327/2670] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:40:37 2022] DELETE /identity/v3/users/2da2deba507f476781f80477f0714666 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:38.981421 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efe48a-904c-44c3-9e1f-beb94699ccbc admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:39.051946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:39.080078 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efe48a-904c-44c3-9e1f-beb94699ccbc admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:39.160627 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3362c1-3bff-4c1b-b0f5-f45faa4c4eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:39.161358 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3362c1-3bff-4c1b-b0f5-f45faa4c4eb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:39.161640 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3362c1-3bff-4c1b-b0f5-f45faa4c4eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:39.161909 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3362c1-3bff-4c1b-b0f5-f45faa4c4eb5 admin admin] PATH_INFO: `/v3/projects/3ce5829f19a242c680eb1ecb6f06f3fa` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:39.163246 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3362c1-3bff-4c1b-b0f5-f45faa4c4eb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ce5829f19a242c680eb1ecb6f06f3fa)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:39.165430 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3362c1-3bff-4c1b-b0f5-f45faa4c4eb5 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:39.289896 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3f932-4e0c-4e4b-af1a-0895477ede81 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:39.290491 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3f932-4e0c-4e4b-af1a-0895477ede81 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:39.290491 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3f932-4e0c-4e4b-af1a-0895477ede81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:39.302918 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cb3362c1-3bff-4c1b-b0f5-f45faa4c4eb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ce5829f19a242c680eb1ecb6f06f3fa', 'request_id': 'req-cb3362c1-3bff-4c1b-b0f5-f45faa4c4eb5'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:39.326183 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-43f3f932-4e0c-4e4b-af1a-0895477ede81 None None] MFA Rules not processed for user `4cddd65cf179486abe179b2382b0d5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:39.327538 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-43f3f932-4e0c-4e4b-af1a-0895477ede81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:39.371542 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cb3362c1-3bff-4c1b-b0f5-f45faa4c4eb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ce5829f19a242c680eb1ecb6f06f3fa', 'request_id': 'req-cb3362c1-3bff-4c1b-b0f5-f45faa4c4eb5'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:39.379422 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cb3362c1-3bff-4c1b-b0f5-f45faa4c4eb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ce5829f19a242c680eb1ecb6f06f3fa', 'request_id': 'req-cb3362c1-3bff-4c1b-b0f5-f45faa4c4eb5'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:39.392465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cb3362c1-3bff-4c1b-b0f5-f45faa4c4eb5 admin admin] The token cache is being invalidate because project 3ce5829f19a242c680eb1ecb6f06f3fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:40:39.392779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cb3362c1-3bff-4c1b-b0f5-f45faa4c4eb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:39.395187 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cb3362c1-3bff-4c1b-b0f5-f45faa4c4eb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:39.398166 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1344/2671] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:40:38 2022] DELETE /identity/v3/projects/3ce5829f19a242c680eb1ecb6f06f3fa => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:40:39.401090 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3362c1-3bff-4c1b-b0f5-f45faa4c4eb5 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:39.404982 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-cb3362c1-3bff-4c1b-b0f5-f45faa4c4eb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:39.432543 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1328/2672] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:39 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:40:39.450894 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f3f932-4e0c-4e4b-af1a-0895477ede81 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:39.453403 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-43f3f932-4e0c-4e4b-af1a-0895477ede81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:39.503967 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3362c1-3bff-4c1b-b0f5-f45faa4c4eb5 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:39.511702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2f61b-33c3-4807-a8a9-5139e80ba66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:39.512605 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2f61b-33c3-4807-a8a9-5139e80ba66e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:39.513023 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2f61b-33c3-4807-a8a9-5139e80ba66e admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:39.513440 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2f61b-33c3-4807-a8a9-5139e80ba66e admin admin] PATH_INFO: `/v3/users/d598bc3cd9f342ff8a321bffdb1359a0` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:39.515204 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e2f61b-33c3-4807-a8a9-5139e80ba66e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d598bc3cd9f342ff8a321bffdb1359a0)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:39.518109 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e2f61b-33c3-4807-a8a9-5139e80ba66e admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:39.518823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f3f932-4e0c-4e4b-af1a-0895477ede81 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:39.522658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed33997-a12c-429e-a072-01766d8036c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:39.523318 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed33997-a12c-429e-a072-01766d8036c6 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:39.523585 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed33997-a12c-429e-a072-01766d8036c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:39.523886 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed33997-a12c-429e-a072-01766d8036c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:39.525736 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-2ed33997-a12c-429e-a072-01766d8036c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:39.540864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:39.584453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:39.588370 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed33997-a12c-429e-a072-01766d8036c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:39.589815 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed33997-a12c-429e-a072-01766d8036c6 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:39.599832 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1329/2673] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:40:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:40:39.729116 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-66e2f61b-33c3-4807-a8a9-5139e80ba66e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd598bc3cd9f342ff8a321bffdb1359a0'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:39.741100 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-66e2f61b-33c3-4807-a8a9-5139e80ba66e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd598bc3cd9f342ff8a321bffdb1359a0'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:39.747820 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-66e2f61b-33c3-4807-a8a9-5139e80ba66e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd598bc3cd9f342ff8a321bffdb1359a0'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:39.811382 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-66e2f61b-33c3-4807-a8a9-5139e80ba66e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd598bc3cd9f342ff8a321bffdb1359a0'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:39.812000 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-66e2f61b-33c3-4807-a8a9-5139e80ba66e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd598bc3cd9f342ff8a321bffdb1359a0'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:39.819320 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1345/2674] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:40:39 2022] DELETE /identity/v3/users/d598bc3cd9f342ff8a321bffdb1359a0 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:40:40.423535 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed33997-a12c-429e-a072-01766d8036c6 service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:40.445923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed33997-a12c-429e-a072-01766d8036c6 service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:40.547086 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2876b42a-be2b-4b63-9085-5285835dc5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:40.547086 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876b42a-be2b-4b63-9085-5285835dc5fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:40.547086 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876b42a-be2b-4b63-9085-5285835dc5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:40.547086 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876b42a-be2b-4b63-9085-5285835dc5fb admin admin] PATH_INFO: `/v3/projects/7ca0dfd1456e4a2590f3a639b0e0ced3` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:40.547521 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2876b42a-be2b-4b63-9085-5285835dc5fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ca0dfd1456e4a2590f3a639b0e0ced3)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:40.547858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2876b42a-be2b-4b63-9085-5285835dc5fb admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:40.674226 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2876b42a-be2b-4b63-9085-5285835dc5fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ca0dfd1456e4a2590f3a639b0e0ced3', 'request_id': 'req-2876b42a-be2b-4b63-9085-5285835dc5fb'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:40.682442 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2876b42a-be2b-4b63-9085-5285835dc5fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ca0dfd1456e4a2590f3a639b0e0ced3', 'request_id': 'req-2876b42a-be2b-4b63-9085-5285835dc5fb'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:40.688349 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2876b42a-be2b-4b63-9085-5285835dc5fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ca0dfd1456e4a2590f3a639b0e0ced3', 'request_id': 'req-2876b42a-be2b-4b63-9085-5285835dc5fb'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:40.748034 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2876b42a-be2b-4b63-9085-5285835dc5fb admin admin] The token cache is being invalidate because project 7ca0dfd1456e4a2590f3a639b0e0ced3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:40:40.748325 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2876b42a-be2b-4b63-9085-5285835dc5fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:40.748974 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2876b42a-be2b-4b63-9085-5285835dc5fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:40:40.751012 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1330/2675] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:40:40 2022] DELETE /identity/v3/projects/7ca0dfd1456e4a2590f3a639b0e0ced3 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:40.757362 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7ef88-9e2e-42b4-9910-11d635f2379b None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:40.757910 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7ef88-9e2e-42b4-9910-11d635f2379b None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:40.757910 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7ef88-9e2e-42b4-9910-11d635f2379b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:40.779906 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-47d7ef88-9e2e-42b4-9910-11d635f2379b None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:40.781114 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-47d7ef88-9e2e-42b4-9910-11d635f2379b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:40.852589 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1346/2676] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:40:40.861681 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d86804-064f-4aed-8cb9-9d17318aa622 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:40.861959 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d86804-064f-4aed-8cb9-9d17318aa622 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:40.862227 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d86804-064f-4aed-8cb9-9d17318aa622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:40.889536 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-93d86804-064f-4aed-8cb9-9d17318aa622 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:40.892269 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-93d86804-064f-4aed-8cb9-9d17318aa622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:40.901870 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1331/2677] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:40:40.906869 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7ef88-9e2e-42b4-9910-11d635f2379b None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:40.917008 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7ef88-9e2e-42b4-9910-11d635f2379b None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:40.922038 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c41a10-8595-4be4-9724-c54240457790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:40.922691 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c41a10-8595-4be4-9724-c54240457790 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:40.922966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c41a10-8595-4be4-9724-c54240457790 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:40.923235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c41a10-8595-4be4-9724-c54240457790 admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:40.923890 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c41a10-8595-4be4-9724-c54240457790 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:40:40.924190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c41a10-8595-4be4-9724-c54240457790 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:40.925797 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c41a10-8595-4be4-9724-c54240457790 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:40.939586 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1347/2678] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:40:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:40:40.944688 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d86804-064f-4aed-8cb9-9d17318aa622 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:40.955303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d86804-064f-4aed-8cb9-9d17318aa622 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:40.960530 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e54aa0-1bde-426c-9c30-43a34a3029b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:40.961164 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e54aa0-1bde-426c-9c30-43a34a3029b2 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:40.961435 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e54aa0-1bde-426c-9c30-43a34a3029b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:40.961700 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e54aa0-1bde-426c-9c30-43a34a3029b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:40.962331 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e54aa0-1bde-426c-9c30-43a34a3029b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:40.964255 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e54aa0-1bde-426c-9c30-43a34a3029b2 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:41.066433 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1332/2679] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:40:40 2022] POST /identity/v3/projects => generated 365 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:40:41.071184 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c41a10-8595-4be4-9724-c54240457790 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:41.083615 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c41a10-8595-4be4-9724-c54240457790 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:41.151110 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa41d7e-cc10-4553-87ce-019aa5bf24f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:41.151631 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa41d7e-cc10-4553-87ce-019aa5bf24f4 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:41.151909 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa41d7e-cc10-4553-87ce-019aa5bf24f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:41.152178 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa41d7e-cc10-4553-87ce-019aa5bf24f4 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:41.152808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa41d7e-cc10-4553-87ce-019aa5bf24f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:41.154744 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa41d7e-cc10-4553-87ce-019aa5bf24f4 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:41.229174 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1348/2680] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:40:41 2022] POST /identity/v3/users => generated 283 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:40:41.238981 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e54aa0-1bde-426c-9c30-43a34a3029b2 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:41.256782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e54aa0-1bde-426c-9c30-43a34a3029b2 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:41.264355 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ada95a-477e-4ebe-8068-beabd140eeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:41.266012 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ada95a-477e-4ebe-8068-beabd140eeb2 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:41.266527 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ada95a-477e-4ebe-8068-beabd140eeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:41.267002 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ada95a-477e-4ebe-8068-beabd140eeb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:41.268136 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ada95a-477e-4ebe-8068-beabd140eeb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:41.271169 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ada95a-477e-4ebe-8068-beabd140eeb2 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:41.290219 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1333/2681] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:40:41.294355 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa41d7e-cc10-4553-87ce-019aa5bf24f4 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:41.306008 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa41d7e-cc10-4553-87ce-019aa5bf24f4 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:41.311295 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af9170-466c-49d5-ace0-8414d2aae044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:41.311949 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af9170-466c-49d5-ace0-8414d2aae044 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:41.312228 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af9170-466c-49d5-ace0-8414d2aae044 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:41.312573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af9170-466c-49d5-ace0-8414d2aae044 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:41.313580 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af9170-466c-49d5-ace0-8414d2aae044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:41.315328 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af9170-466c-49d5-ace0-8414d2aae044 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:41.326731 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1349/2682] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:40:41.332136 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ada95a-477e-4ebe-8068-beabd140eeb2 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:41.342902 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ada95a-477e-4ebe-8068-beabd140eeb2 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:41.348725 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e39ff-c6da-47cf-a12f-1c3932651e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:41.349396 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e39ff-c6da-47cf-a12f-1c3932651e22 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:41.349650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e39ff-c6da-47cf-a12f-1c3932651e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:41.349919 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e39ff-c6da-47cf-a12f-1c3932651e22 admin admin] PATH_INFO: `/v3/projects/bed5e9580fd74fe690c11572487e6525/users/f8e574f9176b4946988c47f074bc9e09/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:41.365895 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e39ff-c6da-47cf-a12f-1c3932651e22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bed5e9580fd74fe690c11572487e6525, user_id=f8e574f9176b4946988c47f074bc9e09, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:41.368582 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e39ff-c6da-47cf-a12f-1c3932651e22 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:41.469828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1334/2683] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:40:41 2022] PUT /identity/v3/projects/bed5e9580fd74fe690c11572487e6525/users/f8e574f9176b4946988c47f074bc9e09/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:40:41.482572 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce1b1c-f557-48dc-8677-ce8c1b2376e0 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:41.482867 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce1b1c-f557-48dc-8677-ce8c1b2376e0 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:41.483170 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce1b1c-f557-48dc-8677-ce8c1b2376e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:41.493638 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:41.518309 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-61ce1b1c-f557-48dc-8677-ce8c1b2376e0 None None] MFA Rules not processed for user `f8e574f9176b4946988c47f074bc9e09`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:40:41.519560 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-61ce1b1c-f557-48dc-8677-ce8c1b2376e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:41.590625 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1350/2684] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:40:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:40:41.603474 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e39ff-c6da-47cf-a12f-1c3932651e22 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:41.605257 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-958e39ff-c6da-47cf-a12f-1c3932651e22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:41.628047 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:41.665908 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e39ff-c6da-47cf-a12f-1c3932651e22 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:41.669753 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ec15d-79e2-4749-a609-b4c56b13e903 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:41.670293 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ec15d-79e2-4749-a609-b4c56b13e903 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:41.670602 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ec15d-79e2-4749-a609-b4c56b13e903 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:41.670875 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ec15d-79e2-4749-a609-b4c56b13e903 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:41.680394 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ec15d-79e2-4749-a609-b4c56b13e903 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:41.681842 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ec15d-79e2-4749-a609-b4c56b13e903 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:41.692246 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1335/2685] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:40:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:40:44.710922 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce1b1c-f557-48dc-8677-ce8c1b2376e0 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:44.712477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-61ce1b1c-f557-48dc-8677-ce8c1b2376e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:44.725279 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce1b1c-f557-48dc-8677-ce8c1b2376e0 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:44.729053 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba821e43-822c-45e9-b4d7-659b97f3028a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:44.729701 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba821e43-822c-45e9-b4d7-659b97f3028a service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:44.729978 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba821e43-822c-45e9-b4d7-659b97f3028a service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:44.730249 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba821e43-822c-45e9-b4d7-659b97f3028a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:44.732041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-ba821e43-822c-45e9-b4d7-659b97f3028a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:44.800601 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba821e43-822c-45e9-b4d7-659b97f3028a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:44.802438 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba821e43-822c-45e9-b4d7-659b97f3028a service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:44.813164 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1351/2686] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:40:44 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:40:51.765509 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=17 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:54.808354 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=61 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:40:58.917736 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ec15d-79e2-4749-a609-b4c56b13e903 service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:40:58.919371 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-010ec15d-79e2-4749-a609-b4c56b13e903 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:58.982416 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ec15d-79e2-4749-a609-b4c56b13e903 service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:40:58.986478 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492889c3-0967-4e02-a3a1-8b51009441c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:40:58.987083 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492889c3-0967-4e02-a3a1-8b51009441c2 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:40:58.987457 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492889c3-0967-4e02-a3a1-8b51009441c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:40:58.987649 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492889c3-0967-4e02-a3a1-8b51009441c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:40:58.989389 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-492889c3-0967-4e02-a3a1-8b51009441c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:40:59.000673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492889c3-0967-4e02-a3a1-8b51009441c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:40:59.002833 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492889c3-0967-4e02-a3a1-8b51009441c2 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:40:59.013107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1336/2687] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:40:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:41:09.008980 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=26 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:41:11.945588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba821e43-822c-45e9-b4d7-659b97f3028a service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:11.946307 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-ba821e43-822c-45e9-b4d7-659b97f3028a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:12.028148 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba821e43-822c-45e9-b4d7-659b97f3028a service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:12.034284 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa409af-3de9-4eeb-9a08-245ad7751754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:12.035217 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa409af-3de9-4eeb-9a08-245ad7751754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:12.035494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa409af-3de9-4eeb-9a08-245ad7751754 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:12.035766 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa409af-3de9-4eeb-9a08-245ad7751754 admin admin] PATH_INFO: `/v3/users/b0527e7febd040c897ab9321598638a9` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:12.037103 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa409af-3de9-4eeb-9a08-245ad7751754 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0527e7febd040c897ab9321598638a9)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:12.039122 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa409af-3de9-4eeb-9a08-245ad7751754 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:12.241254 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-0fa409af-3de9-4eeb-9a08-245ad7751754 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0527e7febd040c897ab9321598638a9'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:12.254137 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-0fa409af-3de9-4eeb-9a08-245ad7751754 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0527e7febd040c897ab9321598638a9'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:12.260941 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-0fa409af-3de9-4eeb-9a08-245ad7751754 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0527e7febd040c897ab9321598638a9'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:12.341598 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-0fa409af-3de9-4eeb-9a08-245ad7751754 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0527e7febd040c897ab9321598638a9'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:12.342345 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-0fa409af-3de9-4eeb-9a08-245ad7751754 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0527e7febd040c897ab9321598638a9'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:12.349511 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1352/2688] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:41:11 2022] DELETE /identity/v3/users/b0527e7febd040c897ab9321598638a9 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:41:12.354104 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6c0ee-f511-4bf2-84af-2c30510081bc None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:12.354685 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6c0ee-f511-4bf2-84af-2c30510081bc None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:12.354778 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6c0ee-f511-4bf2-84af-2c30510081bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:12.375220 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-9db6c0ee-f511-4bf2-84af-2c30510081bc None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:41:12.376483 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-9db6c0ee-f511-4bf2-84af-2c30510081bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:12.440871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1337/2689] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:41:12.451597 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa409af-3de9-4eeb-9a08-245ad7751754 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:12.453155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-0fa409af-3de9-4eeb-9a08-245ad7751754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:12.513864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa409af-3de9-4eeb-9a08-245ad7751754 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:12.518684 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee77b4-1632-4976-927c-3efd6ff47101 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:12.519292 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee77b4-1632-4976-927c-3efd6ff47101 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:12.519573 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee77b4-1632-4976-927c-3efd6ff47101 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:12.519848 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee77b4-1632-4976-927c-3efd6ff47101 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:12.530166 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee77b4-1632-4976-927c-3efd6ff47101 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:12.531816 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee77b4-1632-4976-927c-3efd6ff47101 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:12.541448 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1353/2690] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:41:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:41:13.480974 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db6c0ee-f511-4bf2-84af-2c30510081bc None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:13.483635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-9db6c0ee-f511-4bf2-84af-2c30510081bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:13.499551 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db6c0ee-f511-4bf2-84af-2c30510081bc None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:13.504553 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751d3ba-e021-4bb4-b8b5-8bd099a1b20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:13.506330 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751d3ba-e021-4bb4-b8b5-8bd099a1b20c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:13.506330 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751d3ba-e021-4bb4-b8b5-8bd099a1b20c admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:13.506330 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751d3ba-e021-4bb4-b8b5-8bd099a1b20c admin admin] PATH_INFO: `/v3/projects/ae0704b453a64640901eea2b74ac5c8a` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:13.507918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6751d3ba-e021-4bb4-b8b5-8bd099a1b20c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae0704b453a64640901eea2b74ac5c8a)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:13.509878 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6751d3ba-e021-4bb4-b8b5-8bd099a1b20c admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:13.611693 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-6751d3ba-e021-4bb4-b8b5-8bd099a1b20c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae0704b453a64640901eea2b74ac5c8a', 'request_id': 'req-6751d3ba-e021-4bb4-b8b5-8bd099a1b20c'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:13.618023 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-6751d3ba-e021-4bb4-b8b5-8bd099a1b20c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae0704b453a64640901eea2b74ac5c8a', 'request_id': 'req-6751d3ba-e021-4bb4-b8b5-8bd099a1b20c'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:13.625204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-6751d3ba-e021-4bb4-b8b5-8bd099a1b20c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae0704b453a64640901eea2b74ac5c8a', 'request_id': 'req-6751d3ba-e021-4bb4-b8b5-8bd099a1b20c'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:13.728915 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-6751d3ba-e021-4bb4-b8b5-8bd099a1b20c admin admin] The token cache is being invalidate because project ae0704b453a64640901eea2b74ac5c8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:41:13.729204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-6751d3ba-e021-4bb4-b8b5-8bd099a1b20c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:13.729833 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-6751d3ba-e021-4bb4-b8b5-8bd099a1b20c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:13.731414 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1338/2691] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:41:13 2022] DELETE /identity/v3/projects/ae0704b453a64640901eea2b74ac5c8a => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:41:13.736887 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9566e1-56be-46af-91a0-5b5de9ddd525 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:13.737352 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9566e1-56be-46af-91a0-5b5de9ddd525 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:13.741538 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9566e1-56be-46af-91a0-5b5de9ddd525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:13.758813 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-ab9566e1-56be-46af-91a0-5b5de9ddd525 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:41:13.760108 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-ab9566e1-56be-46af-91a0-5b5de9ddd525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:13.785371 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:41:13.828590 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1354/2692] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:41:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:41:13.836391 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e6466-a75e-4f0c-bc56-11888f6a34ec None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:13.836667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e6466-a75e-4f0c-bc56-11888f6a34ec None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:13.836940 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e6466-a75e-4f0c-bc56-11888f6a34ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:13.859498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-789e6466-a75e-4f0c-bc56-11888f6a34ec None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:41:13.860717 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-789e6466-a75e-4f0c-bc56-11888f6a34ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:13.868819 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1339/2693] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:41:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:41:13.873844 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9566e1-56be-46af-91a0-5b5de9ddd525 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:13.883993 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9566e1-56be-46af-91a0-5b5de9ddd525 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:13.889677 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce15ac7-61fa-4346-abce-6b172a3cd931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:13.890280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce15ac7-61fa-4346-abce-6b172a3cd931 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:13.890601 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce15ac7-61fa-4346-abce-6b172a3cd931 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:13.890876 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce15ac7-61fa-4346-abce-6b172a3cd931 admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:13.891636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce15ac7-61fa-4346-abce-6b172a3cd931 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:41:13.891943 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce15ac7-61fa-4346-abce-6b172a3cd931 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:13.893617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce15ac7-61fa-4346-abce-6b172a3cd931 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:13.906156 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1355/2694] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:41:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:41:13.910864 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e6466-a75e-4f0c-bc56-11888f6a34ec None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:13.921213 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e6466-a75e-4f0c-bc56-11888f6a34ec None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:13.925866 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3751920-a53a-49ef-9a47-6dfde6545a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:13.926523 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3751920-a53a-49ef-9a47-6dfde6545a8b admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:13.926804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3751920-a53a-49ef-9a47-6dfde6545a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:13.927071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3751920-a53a-49ef-9a47-6dfde6545a8b admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:13.927708 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3751920-a53a-49ef-9a47-6dfde6545a8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:13.929591 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3751920-a53a-49ef-9a47-6dfde6545a8b admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:14.043776 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1340/2695] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:41:13 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:41:14.046201 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce15ac7-61fa-4346-abce-6b172a3cd931 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:14.059319 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce15ac7-61fa-4346-abce-6b172a3cd931 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:14.121130 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276232ed-7959-4c9b-b473-a2b82ed9a587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:14.121732 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276232ed-7959-4c9b-b473-a2b82ed9a587 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:14.122012 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276232ed-7959-4c9b-b473-a2b82ed9a587 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:14.123018 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276232ed-7959-4c9b-b473-a2b82ed9a587 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:14.123664 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276232ed-7959-4c9b-b473-a2b82ed9a587 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:14.125592 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276232ed-7959-4c9b-b473-a2b82ed9a587 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:14.223835 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1356/2696] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:41:14 2022] POST /identity/v3/users => generated 292 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:41:14.228047 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3751920-a53a-49ef-9a47-6dfde6545a8b admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:14.239064 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3751920-a53a-49ef-9a47-6dfde6545a8b admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:14.243945 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ae78b-e14d-4004-bc4e-e226cd1bf059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:14.245357 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ae78b-e14d-4004-bc4e-e226cd1bf059 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:14.245357 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ae78b-e14d-4004-bc4e-e226cd1bf059 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:14.245357 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ae78b-e14d-4004-bc4e-e226cd1bf059 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:14.245846 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ae78b-e14d-4004-bc4e-e226cd1bf059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:14.247304 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ae78b-e14d-4004-bc4e-e226cd1bf059 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:14.258717 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1341/2697] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:41:14.263364 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276232ed-7959-4c9b-b473-a2b82ed9a587 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:14.274440 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276232ed-7959-4c9b-b473-a2b82ed9a587 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:14.280116 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58043edc-f4a7-4071-8084-1669c841b674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:14.280722 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58043edc-f4a7-4071-8084-1669c841b674 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:14.281021 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58043edc-f4a7-4071-8084-1669c841b674 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:14.281311 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58043edc-f4a7-4071-8084-1669c841b674 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:14.281953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58043edc-f4a7-4071-8084-1669c841b674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:14.283593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58043edc-f4a7-4071-8084-1669c841b674 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:14.294757 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1357/2698] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:41:14.299070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ae78b-e14d-4004-bc4e-e226cd1bf059 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:14.309265 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ae78b-e14d-4004-bc4e-e226cd1bf059 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:14.314006 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063b865-dcb1-49e2-a2b3-375158abd5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:14.314686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063b865-dcb1-49e2-a2b3-375158abd5a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:14.314961 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063b865-dcb1-49e2-a2b3-375158abd5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:14.315262 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063b865-dcb1-49e2-a2b3-375158abd5a5 admin admin] PATH_INFO: `/v3/projects/2043229f537c4515910f9a4dd1792cd4/users/742d7b534eaf4cbb8455324dbc037032/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:14.331374 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c063b865-dcb1-49e2-a2b3-375158abd5a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2043229f537c4515910f9a4dd1792cd4, user_id=742d7b534eaf4cbb8455324dbc037032, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:14.334146 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c063b865-dcb1-49e2-a2b3-375158abd5a5 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:14.438612 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1342/2699] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:41:14 2022] PUT /identity/v3/projects/2043229f537c4515910f9a4dd1792cd4/users/742d7b534eaf4cbb8455324dbc037032/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:41:14.449126 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ed457-f10b-454d-8d6e-e3f8b852ad7f None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:14.449407 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ed457-f10b-454d-8d6e-e3f8b852ad7f None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:14.449680 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ed457-f10b-454d-8d6e-e3f8b852ad7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:14.482921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-530ed457-f10b-454d-8d6e-e3f8b852ad7f None None] MFA Rules not processed for user `742d7b534eaf4cbb8455324dbc037032`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:41:14.484155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-530ed457-f10b-454d-8d6e-e3f8b852ad7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:14.554222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1358/2700] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:41:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:41:14.566925 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063b865-dcb1-49e2-a2b3-375158abd5a5 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:14.568835 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-c063b865-dcb1-49e2-a2b3-375158abd5a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:14.596159 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:41:14.647138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063b865-dcb1-49e2-a2b3-375158abd5a5 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:14.652104 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b80cb-3131-49db-86e7-2d8ffa589dbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:14.656837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b80cb-3131-49db-86e7-2d8ffa589dbf service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:14.656837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b80cb-3131-49db-86e7-2d8ffa589dbf service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:14.656837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b80cb-3131-49db-86e7-2d8ffa589dbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:14.662687 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b80cb-3131-49db-86e7-2d8ffa589dbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:14.664251 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b80cb-3131-49db-86e7-2d8ffa589dbf service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:14.674637 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1343/2701] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:41:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:41:17.035877 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ed457-f10b-454d-8d6e-e3f8b852ad7f None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:17.037991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-530ed457-f10b-454d-8d6e-e3f8b852ad7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:17.071871 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:41:17.138428 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ed457-f10b-454d-8d6e-e3f8b852ad7f None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:17.143406 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932edbc-ddbf-429e-890b-583b70b327df service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3db46a4b4e74cc0bb2a7bd6f0e99105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:17.144009 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932edbc-ddbf-429e-890b-583b70b327df service placement] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:17.144419 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932edbc-ddbf-429e-890b-583b70b327df service placement] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:17.144771 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932edbc-ddbf-429e-890b-583b70b327df service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:17.146567 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-1932edbc-ddbf-429e-890b-583b70b327df service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:17.159025 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1932edbc-ddbf-429e-890b-583b70b327df service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:17.160829 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1932edbc-ddbf-429e-890b-583b70b327df service placement] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:17.171780 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1359/2702] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:41:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:41:17.773807 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b80cb-3131-49db-86e7-2d8ffa589dbf service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:17.792406 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b80cb-3131-49db-86e7-2d8ffa589dbf service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:17.799049 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28edd503-4440-4a41-b2de-8f420ad320be service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3db46a4b4e74cc0bb2a7bd6f0e99105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:17.799663 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28edd503-4440-4a41-b2de-8f420ad320be service placement] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:17.799967 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28edd503-4440-4a41-b2de-8f420ad320be service placement] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:17.800241 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28edd503-4440-4a41-b2de-8f420ad320be service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:17.802191 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-28edd503-4440-4a41-b2de-8f420ad320be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:17.815892 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28edd503-4440-4a41-b2de-8f420ad320be service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:17.818142 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28edd503-4440-4a41-b2de-8f420ad320be service placement] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:17.830131 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1344/2703] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:41:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:41:17.935349 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932edbc-ddbf-429e-890b-583b70b327df service placement] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:17.936546 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-1932edbc-ddbf-429e-890b-583b70b327df service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:18.019052 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932edbc-ddbf-429e-890b-583b70b327df service placement] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:18.023821 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2d418-b0cc-4510-b60f-34c64fb69868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:18.024528 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2d418-b0cc-4510-b60f-34c64fb69868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:18.024813 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2d418-b0cc-4510-b60f-34c64fb69868 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:18.025090 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2d418-b0cc-4510-b60f-34c64fb69868 admin admin] PATH_INFO: `/v3/users/0918a457ebf34927854aec9af83eb9de` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:18.026228 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2d418-b0cc-4510-b60f-34c64fb69868 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0918a457ebf34927854aec9af83eb9de)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:18.028926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2d418-b0cc-4510-b60f-34c64fb69868 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:18.223901 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-b4b2d418-b0cc-4510-b60f-34c64fb69868 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0918a457ebf34927854aec9af83eb9de'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:18.236788 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-b4b2d418-b0cc-4510-b60f-34c64fb69868 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0918a457ebf34927854aec9af83eb9de'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:18.245083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-b4b2d418-b0cc-4510-b60f-34c64fb69868 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0918a457ebf34927854aec9af83eb9de'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:18.341011 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-b4b2d418-b0cc-4510-b60f-34c64fb69868 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0918a457ebf34927854aec9af83eb9de'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:18.341652 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-b4b2d418-b0cc-4510-b60f-34c64fb69868 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0918a457ebf34927854aec9af83eb9de'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:18.348286 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1360/2704] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:41:17 2022] DELETE /identity/v3/users/0918a457ebf34927854aec9af83eb9de => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:41:18.610613 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28edd503-4440-4a41-b2de-8f420ad320be service placement] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:18.621830 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28edd503-4440-4a41-b2de-8f420ad320be service placement] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:18.683689 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818937c2-7877-4a26-8f80-bdde055a622e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:18.685071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818937c2-7877-4a26-8f80-bdde055a622e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:18.685363 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818937c2-7877-4a26-8f80-bdde055a622e admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:18.685625 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818937c2-7877-4a26-8f80-bdde055a622e admin admin] PATH_INFO: `/v3/projects/3d62e1f2f3ca4bc19e8f7787d8157d5b` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:18.686797 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818937c2-7877-4a26-8f80-bdde055a622e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d62e1f2f3ca4bc19e8f7787d8157d5b)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:18.688688 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818937c2-7877-4a26-8f80-bdde055a622e admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:18.829734 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-818937c2-7877-4a26-8f80-bdde055a622e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d62e1f2f3ca4bc19e8f7787d8157d5b', 'request_id': 'req-818937c2-7877-4a26-8f80-bdde055a622e'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:18.837308 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-818937c2-7877-4a26-8f80-bdde055a622e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d62e1f2f3ca4bc19e8f7787d8157d5b', 'request_id': 'req-818937c2-7877-4a26-8f80-bdde055a622e'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:18.845505 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-818937c2-7877-4a26-8f80-bdde055a622e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d62e1f2f3ca4bc19e8f7787d8157d5b', 'request_id': 'req-818937c2-7877-4a26-8f80-bdde055a622e'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:18.917722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-818937c2-7877-4a26-8f80-bdde055a622e admin admin] The token cache is being invalidate because project 3d62e1f2f3ca4bc19e8f7787d8157d5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:41:18.918116 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-818937c2-7877-4a26-8f80-bdde055a622e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:18.919355 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-818937c2-7877-4a26-8f80-bdde055a622e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:18.920869 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1345/2705] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:41:18 2022] DELETE /identity/v3/projects/3d62e1f2f3ca4bc19e8f7787d8157d5b => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:41:18.924968 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2d418-b0cc-4510-b60f-34c64fb69868 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:18.926358 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-b4b2d418-b0cc-4510-b60f-34c64fb69868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:18.941672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2d418-b0cc-4510-b60f-34c64fb69868 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:18.984081 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:41:18.997491 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818937c2-7877-4a26-8f80-bdde055a622e admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:18.999036 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-818937c2-7877-4a26-8f80-bdde055a622e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:19.024577 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192532d-7c84-4873-a006-1dc4152583b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:19.025225 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192532d-7c84-4873-a006-1dc4152583b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:19.025537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192532d-7c84-4873-a006-1dc4152583b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:19.025963 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192532d-7c84-4873-a006-1dc4152583b1 admin admin] PATH_INFO: `/v3/users/4cddd65cf179486abe179b2382b0d5b4` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:19.027466 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c192532d-7c84-4873-a006-1dc4152583b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cddd65cf179486abe179b2382b0d5b4)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:19.030320 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c192532d-7c84-4873-a006-1dc4152583b1 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:19.051224 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:41:19.074375 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818937c2-7877-4a26-8f80-bdde055a622e admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:19.080038 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986477d1-9180-479a-8f14-fb66460837bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:19.080662 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986477d1-9180-479a-8f14-fb66460837bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:19.080960 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986477d1-9180-479a-8f14-fb66460837bc admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:19.081302 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986477d1-9180-479a-8f14-fb66460837bc admin admin] PATH_INFO: `/v3/users/742d7b534eaf4cbb8455324dbc037032` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:19.082634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986477d1-9180-479a-8f14-fb66460837bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=742d7b534eaf4cbb8455324dbc037032)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:19.084726 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986477d1-9180-479a-8f14-fb66460837bc admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:19.323257 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c192532d-7c84-4873-a006-1dc4152583b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cddd65cf179486abe179b2382b0d5b4'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:19.337870 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c192532d-7c84-4873-a006-1dc4152583b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cddd65cf179486abe179b2382b0d5b4'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:19.345442 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c192532d-7c84-4873-a006-1dc4152583b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cddd65cf179486abe179b2382b0d5b4'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:19.350359 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-986477d1-9180-479a-8f14-fb66460837bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '742d7b534eaf4cbb8455324dbc037032'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:19.493793 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c192532d-7c84-4873-a006-1dc4152583b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cddd65cf179486abe179b2382b0d5b4'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:19.494414 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c192532d-7c84-4873-a006-1dc4152583b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cddd65cf179486abe179b2382b0d5b4'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:19.501091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1361/2706] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:41:18 2022] DELETE /identity/v3/users/4cddd65cf179486abe179b2382b0d5b4 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:41:19.558310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-986477d1-9180-479a-8f14-fb66460837bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '742d7b534eaf4cbb8455324dbc037032'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:19.559012 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-986477d1-9180-479a-8f14-fb66460837bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '742d7b534eaf4cbb8455324dbc037032'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:19.559717 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-986477d1-9180-479a-8f14-fb66460837bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '742d7b534eaf4cbb8455324dbc037032'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:19.573080 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-986477d1-9180-479a-8f14-fb66460837bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '742d7b534eaf4cbb8455324dbc037032'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:19.587044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1346/2707] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:41:18 2022] DELETE /identity/v3/users/742d7b534eaf4cbb8455324dbc037032 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:41:19.592731 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c9a14-63a1-4455-99c4-2481e855aea8 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:19.593241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c9a14-63a1-4455-99c4-2481e855aea8 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:19.593564 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c9a14-63a1-4455-99c4-2481e855aea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:19.615013 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-e00c9a14-63a1-4455-99c4-2481e855aea8 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:41:19.616237 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-e00c9a14-63a1-4455-99c4-2481e855aea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:19.680020 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1362/2708] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:41:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:41:19.690432 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986477d1-9180-479a-8f14-fb66460837bc admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:19.691962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-986477d1-9180-479a-8f14-fb66460837bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:19.753943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986477d1-9180-479a-8f14-fb66460837bc admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:19.757658 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00193a52-4ab6-42f3-89ad-9b59d7dda007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:19.758268 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00193a52-4ab6-42f3-89ad-9b59d7dda007 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:19.758573 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00193a52-4ab6-42f3-89ad-9b59d7dda007 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:19.758868 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00193a52-4ab6-42f3-89ad-9b59d7dda007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:19.765957 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c9a14-63a1-4455-99c4-2481e855aea8 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:19.769516 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-e00c9a14-63a1-4455-99c4-2481e855aea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:19.771466 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00193a52-4ab6-42f3-89ad-9b59d7dda007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:19.776689 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00193a52-4ab6-42f3-89ad-9b59d7dda007 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:19.784797 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1347/2709] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:41:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:41:19.794638 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c9a14-63a1-4455-99c4-2481e855aea8 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:19.800812 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93050a0-4e3c-483f-859c-66b9fa6fbde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:19.801521 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93050a0-4e3c-483f-859c-66b9fa6fbde0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:19.801713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93050a0-4e3c-483f-859c-66b9fa6fbde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:19.802025 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93050a0-4e3c-483f-859c-66b9fa6fbde0 admin admin] PATH_INFO: `/v3/projects/1f43f730f3e041ada9987e59d235a582` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:19.803443 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93050a0-4e3c-483f-859c-66b9fa6fbde0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f43f730f3e041ada9987e59d235a582)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:19.805459 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93050a0-4e3c-483f-859c-66b9fa6fbde0 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:19.947470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:41:19.954459 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c93050a0-4e3c-483f-859c-66b9fa6fbde0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f43f730f3e041ada9987e59d235a582', 'request_id': 'req-c93050a0-4e3c-483f-859c-66b9fa6fbde0'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:20.055048 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c93050a0-4e3c-483f-859c-66b9fa6fbde0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f43f730f3e041ada9987e59d235a582', 'request_id': 'req-c93050a0-4e3c-483f-859c-66b9fa6fbde0'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:20.066789 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c93050a0-4e3c-483f-859c-66b9fa6fbde0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f43f730f3e041ada9987e59d235a582', 'request_id': 'req-c93050a0-4e3c-483f-859c-66b9fa6fbde0'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:20.080443 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c93050a0-4e3c-483f-859c-66b9fa6fbde0 admin admin] The token cache is being invalidate because project 1f43f730f3e041ada9987e59d235a582 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:41:20.080736 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c93050a0-4e3c-483f-859c-66b9fa6fbde0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:20.081563 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c93050a0-4e3c-483f-859c-66b9fa6fbde0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:20.083198 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1363/2710] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:41:19 2022] DELETE /identity/v3/projects/1f43f730f3e041ada9987e59d235a582 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:41:20.088354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845cd6ea-ae5c-4688-9469-2f21234f1d1f None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:20.088964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845cd6ea-ae5c-4688-9469-2f21234f1d1f None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:20.088964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845cd6ea-ae5c-4688-9469-2f21234f1d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:20.115351 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-845cd6ea-ae5c-4688-9469-2f21234f1d1f None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:41:20.116659 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-845cd6ea-ae5c-4688-9469-2f21234f1d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:20.179423 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1348/2711] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:41:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:41:20.188177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a71761-e273-41f8-8117-c161a66507a7 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:20.188492 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a71761-e273-41f8-8117-c161a66507a7 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:20.188770 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a71761-e273-41f8-8117-c161a66507a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:20.209485 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-90a71761-e273-41f8-8117-c161a66507a7 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:41:20.210872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-90a71761-e273-41f8-8117-c161a66507a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:20.219328 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1364/2712] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:41:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:41:20.227385 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845cd6ea-ae5c-4688-9469-2f21234f1d1f None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:20.233365 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:41:20.239010 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845cd6ea-ae5c-4688-9469-2f21234f1d1f None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:20.243762 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8bde26-46c3-4ed0-967e-9688ea6f0440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:20.244385 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8bde26-46c3-4ed0-967e-9688ea6f0440 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:20.250354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8bde26-46c3-4ed0-967e-9688ea6f0440 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:20.250354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8bde26-46c3-4ed0-967e-9688ea6f0440 admin admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:20.250354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8bde26-46c3-4ed0-967e-9688ea6f0440 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:41:20.250354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8bde26-46c3-4ed0-967e-9688ea6f0440 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:20.250354 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8bde26-46c3-4ed0-967e-9688ea6f0440 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:20.261593 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1349/2713] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:41:20.266129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a71761-e273-41f8-8117-c161a66507a7 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:20.273440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8bde26-46c3-4ed0-967e-9688ea6f0440 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:20.275409 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-3f8bde26-46c3-4ed0-967e-9688ea6f0440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:20.276249 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a71761-e273-41f8-8117-c161a66507a7 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:20.281241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ac167-012f-4794-b5d7-173bb8f82049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:20.281865 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ac167-012f-4794-b5d7-173bb8f82049 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:20.284344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ac167-012f-4794-b5d7-173bb8f82049 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:20.284344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ac167-012f-4794-b5d7-173bb8f82049 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:20.284344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ac167-012f-4794-b5d7-173bb8f82049 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:20.285374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ac167-012f-4794-b5d7-173bb8f82049 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:20.290481 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8bde26-46c3-4ed0-967e-9688ea6f0440 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:20.297422 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bcba26-20af-4bc5-ac4c-0d75d268582c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:20.298238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcba26-20af-4bc5-ac4c-0d75d268582c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:20.298802 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcba26-20af-4bc5-ac4c-0d75d268582c admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:20.299146 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcba26-20af-4bc5-ac4c-0d75d268582c admin admin] PATH_INFO: `/v3/projects/2043229f537c4515910f9a4dd1792cd4` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:20.301108 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bcba26-20af-4bc5-ac4c-0d75d268582c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2043229f537c4515910f9a4dd1792cd4)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:20.303549 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bcba26-20af-4bc5-ac4c-0d75d268582c admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:20.480371 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1365/2714] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:41:20 2022] POST /identity/v3/projects => generated 375 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:41:20.485403 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ac167-012f-4794-b5d7-173bb8f82049 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:20.538212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-72bcba26-20af-4bc5-ac4c-0d75d268582c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2043229f537c4515910f9a4dd1792cd4', 'request_id': 'req-72bcba26-20af-4bc5-ac4c-0d75d268582c'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:20.546296 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-72bcba26-20af-4bc5-ac4c-0d75d268582c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2043229f537c4515910f9a4dd1792cd4', 'request_id': 'req-72bcba26-20af-4bc5-ac4c-0d75d268582c'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:20.552372 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-72bcba26-20af-4bc5-ac4c-0d75d268582c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2043229f537c4515910f9a4dd1792cd4', 'request_id': 'req-72bcba26-20af-4bc5-ac4c-0d75d268582c'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:20.558154 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ac167-012f-4794-b5d7-173bb8f82049 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:20.619765 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4369c54-3789-4746-b7ac-91a5df7ba480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:20.620330 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4369c54-3789-4746-b7ac-91a5df7ba480 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:20.620604 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4369c54-3789-4746-b7ac-91a5df7ba480 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:20.620871 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4369c54-3789-4746-b7ac-91a5df7ba480 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:20.621499 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4369c54-3789-4746-b7ac-91a5df7ba480 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:20.624082 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4369c54-3789-4746-b7ac-91a5df7ba480 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:20.638233 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-72bcba26-20af-4bc5-ac4c-0d75d268582c admin admin] The token cache is being invalidate because project 2043229f537c4515910f9a4dd1792cd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:41:20.638554 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-72bcba26-20af-4bc5-ac4c-0d75d268582c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:20.639188 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-72bcba26-20af-4bc5-ac4c-0d75d268582c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:20.640715 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1350/2715] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:41:20 2022] DELETE /identity/v3/projects/2043229f537c4515910f9a4dd1792cd4 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:41:20.747512 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1366/2716] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:41:20 2022] POST /identity/v3/users => generated 288 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:41:20.751673 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bcba26-20af-4bc5-ac4c-0d75d268582c admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:20.753247 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-72bcba26-20af-4bc5-ac4c-0d75d268582c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:20.767015 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bcba26-20af-4bc5-ac4c-0d75d268582c admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:20.771722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631c9a9-529a-4bee-aba1-6f33cd05e4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:20.775860 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631c9a9-529a-4bee-aba1-6f33cd05e4d2 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:20.775860 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631c9a9-529a-4bee-aba1-6f33cd05e4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:20.775860 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631c9a9-529a-4bee-aba1-6f33cd05e4d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:20.775860 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631c9a9-529a-4bee-aba1-6f33cd05e4d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:20.775860 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631c9a9-529a-4bee-aba1-6f33cd05e4d2 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:20.787550 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1351/2717] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:41:20.792896 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4369c54-3789-4746-b7ac-91a5df7ba480 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:20.806897 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4369c54-3789-4746-b7ac-91a5df7ba480 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:20.812295 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52768a-7c0b-4667-8bb7-79cf502678b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:20.812915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52768a-7c0b-4667-8bb7-79cf502678b2 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:20.813192 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52768a-7c0b-4667-8bb7-79cf502678b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:20.813461 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52768a-7c0b-4667-8bb7-79cf502678b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:20.814111 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e52768a-7c0b-4667-8bb7-79cf502678b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:20.815771 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e52768a-7c0b-4667-8bb7-79cf502678b2 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:20.828365 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1367/2718] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:41:20.833178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631c9a9-529a-4bee-aba1-6f33cd05e4d2 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:20.846138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631c9a9-529a-4bee-aba1-6f33cd05e4d2 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:20.851930 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41d2d2-a130-4d39-a373-0c299f377545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:20.852723 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41d2d2-a130-4d39-a373-0c299f377545 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:20.853068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41d2d2-a130-4d39-a373-0c299f377545 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:20.853410 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41d2d2-a130-4d39-a373-0c299f377545 admin admin] PATH_INFO: `/v3/projects/9f4879db334d47be89d6f2dad34e5bff/users/4d5573016cf54aa396be0c8ee85e3626/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:20.872648 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a41d2d2-a130-4d39-a373-0c299f377545 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f4879db334d47be89d6f2dad34e5bff, user_id=4d5573016cf54aa396be0c8ee85e3626, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:20.875428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a41d2d2-a130-4d39-a373-0c299f377545 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:20.946555 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1352/2719] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:41:20 2022] PUT /identity/v3/projects/9f4879db334d47be89d6f2dad34e5bff/users/4d5573016cf54aa396be0c8ee85e3626/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:41:20.958101 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082ce4c-1022-4bae-9680-53b4e9cb57f2 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:20.958411 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082ce4c-1022-4bae-9680-53b4e9cb57f2 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:20.958687 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082ce4c-1022-4bae-9680-53b4e9cb57f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:20.988610 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-a082ce4c-1022-4bae-9680-53b4e9cb57f2 None None] MFA Rules not processed for user `4d5573016cf54aa396be0c8ee85e3626`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:41:20.989820 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a082ce4c-1022-4bae-9680-53b4e9cb57f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:21.008099 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:41:21.061414 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1368/2720] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:41:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:41:21.066099 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41d2d2-a130-4d39-a373-0c299f377545 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:21.137532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41d2d2-a130-4d39-a373-0c299f377545 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:21.142367 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27ca04-68cc-4ed1-a84d-c00f5a789a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:21.143008 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27ca04-68cc-4ed1-a84d-c00f5a789a38 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:21.143282 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27ca04-68cc-4ed1-a84d-c00f5a789a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:21.143550 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27ca04-68cc-4ed1-a84d-c00f5a789a38 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:21.144220 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d27ca04-68cc-4ed1-a84d-c00f5a789a38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:21.146093 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d27ca04-68cc-4ed1-a84d-c00f5a789a38 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:21.365893 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1353/2721] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:41:21 2022] POST /identity/v3/projects => generated 373 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:41:21.370101 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082ce4c-1022-4bae-9680-53b4e9cb57f2 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:21.381352 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082ce4c-1022-4bae-9680-53b4e9cb57f2 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:21.443202 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829a07cb-949c-4c71-a716-62627334d28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:21.443817 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a07cb-949c-4c71-a716-62627334d28f admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:21.444093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a07cb-949c-4c71-a716-62627334d28f admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:21.444371 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a07cb-949c-4c71-a716-62627334d28f admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:21.445013 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829a07cb-949c-4c71-a716-62627334d28f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:21.447002 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829a07cb-949c-4c71-a716-62627334d28f admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:21.572412 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1369/2722] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:41:21 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:41:21.574013 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27ca04-68cc-4ed1-a84d-c00f5a789a38 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:21.586823 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27ca04-68cc-4ed1-a84d-c00f5a789a38 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:21.592238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d218b-286e-4dba-9496-f77182233a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:21.592890 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d218b-286e-4dba-9496-f77182233a86 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:21.593135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d218b-286e-4dba-9496-f77182233a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:21.593404 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d218b-286e-4dba-9496-f77182233a86 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:21.594245 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715d218b-286e-4dba-9496-f77182233a86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:21.595995 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715d218b-286e-4dba-9496-f77182233a86 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:21.607949 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1354/2723] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:41:21.612470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829a07cb-949c-4c71-a716-62627334d28f admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:21.623058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829a07cb-949c-4c71-a716-62627334d28f admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:21.627864 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffbe4cc-da41-4d9a-b058-1f1c967b2fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:21.628515 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbe4cc-da41-4d9a-b058-1f1c967b2fb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:21.628884 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbe4cc-da41-4d9a-b058-1f1c967b2fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:21.629158 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbe4cc-da41-4d9a-b058-1f1c967b2fb2 admin admin] PATH_INFO: `/v3/projects/21f457d6650c4395b41397e612e9b382/users/475ef24d45df4a659765520a4e48370c/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:21.646075 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffbe4cc-da41-4d9a-b058-1f1c967b2fb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21f457d6650c4395b41397e612e9b382, user_id=475ef24d45df4a659765520a4e48370c, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:21.649059 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffbe4cc-da41-4d9a-b058-1f1c967b2fb2 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:21.756859 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1370/2724] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:41:21 2022] PUT /identity/v3/projects/21f457d6650c4395b41397e612e9b382/users/475ef24d45df4a659765520a4e48370c/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:41:21.760879 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d218b-286e-4dba-9496-f77182233a86 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:21.771837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d218b-286e-4dba-9496-f77182233a86 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:21.781607 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:41:21.836017 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053977f5-ee21-42d4-8537-ff13fb2db028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:21.836821 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053977f5-ee21-42d4-8537-ff13fb2db028 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:21.837102 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053977f5-ee21-42d4-8537-ff13fb2db028 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:21.837582 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053977f5-ee21-42d4-8537-ff13fb2db028 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:21.838588 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053977f5-ee21-42d4-8537-ff13fb2db028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:21.840208 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053977f5-ee21-42d4-8537-ff13fb2db028 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:21.851353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1355/2725] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:41:21.855744 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffbe4cc-da41-4d9a-b058-1f1c967b2fb2 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:21.866490 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffbe4cc-da41-4d9a-b058-1f1c967b2fb2 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:21.871621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a48198-bf57-456f-abfa-cc81353543e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:21.871822 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a48198-bf57-456f-abfa-cc81353543e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:21.872101 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a48198-bf57-456f-abfa-cc81353543e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:21.872382 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a48198-bf57-456f-abfa-cc81353543e7 admin admin] PATH_INFO: `/v3/projects/21f457d6650c4395b41397e612e9b382/users/475ef24d45df4a659765520a4e48370c/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:21.874433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a48198-bf57-456f-abfa-cc81353543e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21f457d6650c4395b41397e612e9b382, user_id=475ef24d45df4a659765520a4e48370c, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:21.877175 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a48198-bf57-456f-abfa-cc81353543e7 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:21.889801 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1371/2726] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:41:21 2022] PUT /identity/v3/projects/21f457d6650c4395b41397e612e9b382/users/475ef24d45df4a659765520a4e48370c/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:41:21.900067 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbc159-5dc5-4bd9-808e-26b222c485b2 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:21.900356 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbc159-5dc5-4bd9-808e-26b222c485b2 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:21.904070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbc159-5dc5-4bd9-808e-26b222c485b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:21.932054 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-b0dbc159-5dc5-4bd9-808e-26b222c485b2 None None] MFA Rules not processed for user `475ef24d45df4a659765520a4e48370c`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:41:21.933300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-b0dbc159-5dc5-4bd9-808e-26b222c485b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:22.008235 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1356/2727] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:41:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:41:22.019970 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a48198-bf57-456f-abfa-cc81353543e7 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:22.021780 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-19a48198-bf57-456f-abfa-cc81353543e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:22.083257 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a48198-bf57-456f-abfa-cc81353543e7 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:22.087001 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c72aa-f154-4498-bcf3-1d957ebfc7a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:22.087622 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c72aa-f154-4498-bcf3-1d957ebfc7a0 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:22.087927 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c72aa-f154-4498-bcf3-1d957ebfc7a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:22.088322 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c72aa-f154-4498-bcf3-1d957ebfc7a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:22.110019 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:41:22.152294 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276c72aa-f154-4498-bcf3-1d957ebfc7a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:22.157949 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276c72aa-f154-4498-bcf3-1d957ebfc7a0 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:22.163966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1372/2728] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:41:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:41:26.314089 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dbc159-5dc5-4bd9-808e-26b222c485b2 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:26.394267 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dbc159-5dc5-4bd9-808e-26b222c485b2 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:26.399050 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeafab7d-36ee-4a4c-adf6-17b549e1cd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:26.399715 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeafab7d-36ee-4a4c-adf6-17b549e1cd16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:26.399991 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeafab7d-36ee-4a4c-adf6-17b549e1cd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:26.400261 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeafab7d-36ee-4a4c-adf6-17b549e1cd16 admin admin] PATH_INFO: `/v3/users/4d5573016cf54aa396be0c8ee85e3626` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:26.401396 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeafab7d-36ee-4a4c-adf6-17b549e1cd16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d5573016cf54aa396be0c8ee85e3626)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:26.404054 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeafab7d-36ee-4a4c-adf6-17b549e1cd16 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:26.591500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:41:26.600605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-aeafab7d-36ee-4a4c-adf6-17b549e1cd16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d5573016cf54aa396be0c8ee85e3626'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:26.671614 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-aeafab7d-36ee-4a4c-adf6-17b549e1cd16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d5573016cf54aa396be0c8ee85e3626'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:26.672497 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-aeafab7d-36ee-4a4c-adf6-17b549e1cd16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d5573016cf54aa396be0c8ee85e3626'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:26.673478 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-aeafab7d-36ee-4a4c-adf6-17b549e1cd16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d5573016cf54aa396be0c8ee85e3626'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:26.687805 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-aeafab7d-36ee-4a4c-adf6-17b549e1cd16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d5573016cf54aa396be0c8ee85e3626'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:26.700466 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1357/2729] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:41:26 2022] DELETE /identity/v3/users/4d5573016cf54aa396be0c8ee85e3626 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:41:26.705480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e166fb-a494-4548-9741-59fb43125b66 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:26.705976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e166fb-a494-4548-9741-59fb43125b66 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:26.705976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e166fb-a494-4548-9741-59fb43125b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:26.725372 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-c3e166fb-a494-4548-9741-59fb43125b66 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:41:26.726632 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-c3e166fb-a494-4548-9741-59fb43125b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:26.787419 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1373/2730] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:41:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:41:26.797969 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeafab7d-36ee-4a4c-adf6-17b549e1cd16 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:26.799480 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-aeafab7d-36ee-4a4c-adf6-17b549e1cd16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:26.869501 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeafab7d-36ee-4a4c-adf6-17b549e1cd16 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:26.874275 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a345c57-9730-4941-91b7-648969921b58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:26.874938 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a345c57-9730-4941-91b7-648969921b58 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:26.875218 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a345c57-9730-4941-91b7-648969921b58 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:26.875484 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a345c57-9730-4941-91b7-648969921b58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:26.885256 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a345c57-9730-4941-91b7-648969921b58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:26.886717 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a345c57-9730-4941-91b7-648969921b58 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:26.896444 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1358/2731] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:41:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:41:27.290165 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e166fb-a494-4548-9741-59fb43125b66 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:27.291827 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-c3e166fb-a494-4548-9741-59fb43125b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:27.310804 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e166fb-a494-4548-9741-59fb43125b66 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:27.316100 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2cc321-9982-466a-850c-c1a5f3241406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:27.317026 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2cc321-9982-466a-850c-c1a5f3241406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:27.317144 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2cc321-9982-466a-850c-c1a5f3241406 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:27.317442 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2cc321-9982-466a-850c-c1a5f3241406 admin admin] PATH_INFO: `/v3/projects/9f4879db334d47be89d6f2dad34e5bff` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:27.318707 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2cc321-9982-466a-850c-c1a5f3241406 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f4879db334d47be89d6f2dad34e5bff)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:27.320702 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2cc321-9982-466a-850c-c1a5f3241406 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:27.437824 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-fc2cc321-9982-466a-850c-c1a5f3241406 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f4879db334d47be89d6f2dad34e5bff', 'request_id': 'req-fc2cc321-9982-466a-850c-c1a5f3241406'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:27.523387 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-fc2cc321-9982-466a-850c-c1a5f3241406 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f4879db334d47be89d6f2dad34e5bff', 'request_id': 'req-fc2cc321-9982-466a-850c-c1a5f3241406'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:27.530407 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-fc2cc321-9982-466a-850c-c1a5f3241406 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f4879db334d47be89d6f2dad34e5bff', 'request_id': 'req-fc2cc321-9982-466a-850c-c1a5f3241406'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:27.544904 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-fc2cc321-9982-466a-850c-c1a5f3241406 admin admin] The token cache is being invalidate because project 9f4879db334d47be89d6f2dad34e5bff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:41:27.545249 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-fc2cc321-9982-466a-850c-c1a5f3241406 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:27.546122 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-fc2cc321-9982-466a-850c-c1a5f3241406 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:27.547915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1374/2732] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:41:27 2022] DELETE /identity/v3/projects/9f4879db334d47be89d6f2dad34e5bff => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:41:27.552118 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a345c57-9730-4941-91b7-648969921b58 service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:27.563310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a345c57-9730-4941-91b7-648969921b58 service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:27.621775 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f8ea1-365c-4a62-aa5f-055f1235ca0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:27.622348 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f8ea1-365c-4a62-aa5f-055f1235ca0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:27.622733 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f8ea1-365c-4a62-aa5f-055f1235ca0e admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:27.623035 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f8ea1-365c-4a62-aa5f-055f1235ca0e admin admin] PATH_INFO: `/v3/users/475ef24d45df4a659765520a4e48370c` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:27.624214 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f8ea1-365c-4a62-aa5f-055f1235ca0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=475ef24d45df4a659765520a4e48370c)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:27.626247 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f8ea1-365c-4a62-aa5f-055f1235ca0e admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:27.811223 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-a47f8ea1-365c-4a62-aa5f-055f1235ca0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '475ef24d45df4a659765520a4e48370c'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:27.865249 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-a47f8ea1-365c-4a62-aa5f-055f1235ca0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '475ef24d45df4a659765520a4e48370c'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:27.865887 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-a47f8ea1-365c-4a62-aa5f-055f1235ca0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '475ef24d45df4a659765520a4e48370c'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:27.866556 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-a47f8ea1-365c-4a62-aa5f-055f1235ca0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '475ef24d45df4a659765520a4e48370c'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:27.868890 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:41:27.879573 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-a47f8ea1-365c-4a62-aa5f-055f1235ca0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '475ef24d45df4a659765520a4e48370c'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:27.892135 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1359/2733] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:41:27 2022] DELETE /identity/v3/users/475ef24d45df4a659765520a4e48370c => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:41:28.336620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2cc321-9982-466a-850c-c1a5f3241406 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:28.342206 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-fc2cc321-9982-466a-850c-c1a5f3241406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:28.356624 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2cc321-9982-466a-850c-c1a5f3241406 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:28.413673 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:41:28.428905 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f18f53-a2c0-4bdf-acff-79fa2ffc29eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:28.429534 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f18f53-a2c0-4bdf-acff-79fa2ffc29eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:28.429866 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f18f53-a2c0-4bdf-acff-79fa2ffc29eb admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:28.430145 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f18f53-a2c0-4bdf-acff-79fa2ffc29eb admin admin] PATH_INFO: `/v3/projects/21f457d6650c4395b41397e612e9b382` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:28.431334 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f18f53-a2c0-4bdf-acff-79fa2ffc29eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=21f457d6650c4395b41397e612e9b382)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:28.433235 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f18f53-a2c0-4bdf-acff-79fa2ffc29eb admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:28.531034 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-02f18f53-a2c0-4bdf-acff-79fa2ffc29eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21f457d6650c4395b41397e612e9b382', 'request_id': 'req-02f18f53-a2c0-4bdf-acff-79fa2ffc29eb'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:28.591468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-02f18f53-a2c0-4bdf-acff-79fa2ffc29eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21f457d6650c4395b41397e612e9b382', 'request_id': 'req-02f18f53-a2c0-4bdf-acff-79fa2ffc29eb'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:28.597729 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-02f18f53-a2c0-4bdf-acff-79fa2ffc29eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21f457d6650c4395b41397e612e9b382', 'request_id': 'req-02f18f53-a2c0-4bdf-acff-79fa2ffc29eb'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:28.609422 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-02f18f53-a2c0-4bdf-acff-79fa2ffc29eb admin admin] The token cache is being invalidate because project 21f457d6650c4395b41397e612e9b382 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:41:28.609732 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-02f18f53-a2c0-4bdf-acff-79fa2ffc29eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:28.610411 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-02f18f53-a2c0-4bdf-acff-79fa2ffc29eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:28.611922 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1375/2734] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:41:28 2022] DELETE /identity/v3/projects/21f457d6650c4395b41397e612e9b382 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:41:28.616801 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9efac4-845d-40a0-8ca2-3d5a723a47b2 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:28.617274 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9efac4-845d-40a0-8ca2-3d5a723a47b2 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:28.617418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9efac4-845d-40a0-8ca2-3d5a723a47b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:28.638356 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-2b9efac4-845d-40a0-8ca2-3d5a723a47b2 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:41:28.639588 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-2b9efac4-845d-40a0-8ca2-3d5a723a47b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:28.700637 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1360/2735] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:41:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:41:28.709110 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea6bf9-5891-4dae-ac1d-343c44a917fe None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:28.709394 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea6bf9-5891-4dae-ac1d-343c44a917fe None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:28.709668 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea6bf9-5891-4dae-ac1d-343c44a917fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:28.729965 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-e6ea6bf9-5891-4dae-ac1d-343c44a917fe None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:41:28.732026 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-e6ea6bf9-5891-4dae-ac1d-343c44a917fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:28.740029 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1376/2736] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:41:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:41:28.744481 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9efac4-845d-40a0-8ca2-3d5a723a47b2 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:28.755327 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9efac4-845d-40a0-8ca2-3d5a723a47b2 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:28.761232 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f635cbcc-6f9c-4f79-b4ae-68ddb0918243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:28.761704 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635cbcc-6f9c-4f79-b4ae-68ddb0918243 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:28.761980 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635cbcc-6f9c-4f79-b4ae-68ddb0918243 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:28.762432 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635cbcc-6f9c-4f79-b4ae-68ddb0918243 admin admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:28.762952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f635cbcc-6f9c-4f79-b4ae-68ddb0918243 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:41:28.763249 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f635cbcc-6f9c-4f79-b4ae-68ddb0918243 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:28.764867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f635cbcc-6f9c-4f79-b4ae-68ddb0918243 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:28.777145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1361/2737] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:41:28.782324 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea6bf9-5891-4dae-ac1d-343c44a917fe None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:28.792192 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea6bf9-5891-4dae-ac1d-343c44a917fe None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:28.797781 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3965352b-16c0-4936-8a81-9de2f674abee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:28.798442 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965352b-16c0-4936-8a81-9de2f674abee admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:28.798725 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965352b-16c0-4936-8a81-9de2f674abee admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:28.798998 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965352b-16c0-4936-8a81-9de2f674abee admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:28.799665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3965352b-16c0-4936-8a81-9de2f674abee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:28.801677 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3965352b-16c0-4936-8a81-9de2f674abee admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:28.890538 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1377/2738] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:41:28 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:41:28.894652 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f635cbcc-6f9c-4f79-b4ae-68ddb0918243 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:28.905970 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f635cbcc-6f9c-4f79-b4ae-68ddb0918243 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:28.967637 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5655a-f38a-481e-8e00-bf4ba9eaf4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:28.968256 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5655a-f38a-481e-8e00-bf4ba9eaf4aa admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:28.968532 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5655a-f38a-481e-8e00-bf4ba9eaf4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:28.968803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5655a-f38a-481e-8e00-bf4ba9eaf4aa admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:28.969453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d5655a-f38a-481e-8e00-bf4ba9eaf4aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:28.971469 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d5655a-f38a-481e-8e00-bf4ba9eaf4aa admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:29.083727 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1362/2739] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:41:28 2022] POST /identity/v3/users => generated 286 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:41:29.087928 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3965352b-16c0-4936-8a81-9de2f674abee admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:29.099070 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3965352b-16c0-4936-8a81-9de2f674abee admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:29.104330 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b61c86-e60c-4944-ab6f-1521f857f58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:29.104898 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b61c86-e60c-4944-ab6f-1521f857f58b admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:29.105173 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b61c86-e60c-4944-ab6f-1521f857f58b admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:29.106099 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b61c86-e60c-4944-ab6f-1521f857f58b admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:29.106829 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b61c86-e60c-4944-ab6f-1521f857f58b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:29.108470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b61c86-e60c-4944-ab6f-1521f857f58b admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:29.119804 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1378/2740] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:41:29.123963 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5655a-f38a-481e-8e00-bf4ba9eaf4aa admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:29.134620 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5655a-f38a-481e-8e00-bf4ba9eaf4aa admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:29.139991 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b16b5-4b69-4884-a927-9098782260d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:29.140591 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b16b5-4b69-4884-a927-9098782260d9 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:29.140870 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b16b5-4b69-4884-a927-9098782260d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:29.141142 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b16b5-4b69-4884-a927-9098782260d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:29.141817 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2b16b5-4b69-4884-a927-9098782260d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:29.143450 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2b16b5-4b69-4884-a927-9098782260d9 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:29.155956 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1363/2741] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:41:29.160248 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b61c86-e60c-4944-ab6f-1521f857f58b admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:29.171411 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b61c86-e60c-4944-ab6f-1521f857f58b admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:29.176128 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d3c27-fe96-496c-9de3-5a819e8cd7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:29.176845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d3c27-fe96-496c-9de3-5a819e8cd7f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:29.177125 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d3c27-fe96-496c-9de3-5a819e8cd7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:29.177399 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d3c27-fe96-496c-9de3-5a819e8cd7f8 admin admin] PATH_INFO: `/v3/projects/a63507e4eb9f4dafbbb6aa5af971632c/users/4fac6277983e497b86ed29c8f02d8987/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:29.194614 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317d3c27-fe96-496c-9de3-5a819e8cd7f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a63507e4eb9f4dafbbb6aa5af971632c, user_id=4fac6277983e497b86ed29c8f02d8987, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:29.197318 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317d3c27-fe96-496c-9de3-5a819e8cd7f8 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:29.289203 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1379/2742] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:41:29 2022] PUT /identity/v3/projects/a63507e4eb9f4dafbbb6aa5af971632c/users/4fac6277983e497b86ed29c8f02d8987/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:41:29.294474 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629e7ee-7d34-41cf-bd97-aa32232cfa26 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:29.294756 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629e7ee-7d34-41cf-bd97-aa32232cfa26 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:29.295154 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629e7ee-7d34-41cf-bd97-aa32232cfa26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:29.314199 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-9629e7ee-7d34-41cf-bd97-aa32232cfa26 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:41:29.315496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-9629e7ee-7d34-41cf-bd97-aa32232cfa26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:29.350589 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:41:29.376038 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1364/2743] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:41:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:41:29.385752 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d3c27-fe96-496c-9de3-5a819e8cd7f8 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:29.388025 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-317d3c27-fe96-496c-9de3-5a819e8cd7f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:29.449164 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d3c27-fe96-496c-9de3-5a819e8cd7f8 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:29.452919 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a3283-2615-4cae-8241-1d1532b4ef17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:29.453783 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a3283-2615-4cae-8241-1d1532b4ef17 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:29.454086 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a3283-2615-4cae-8241-1d1532b4ef17 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:29.454361 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a3283-2615-4cae-8241-1d1532b4ef17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:29.463793 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a3283-2615-4cae-8241-1d1532b4ef17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:29.465275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a3283-2615-4cae-8241-1d1532b4ef17 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:29.475262 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1380/2744] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:41:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:41:31.405372 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefedf0-4390-4c29-bfd4-860f9fb196fa None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:31.406178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefedf0-4390-4c29-bfd4-860f9fb196fa None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:31.406178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefedf0-4390-4c29-bfd4-860f9fb196fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:31.435923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-1fefedf0-4390-4c29-bfd4-860f9fb196fa None None] MFA Rules not processed for user `4fac6277983e497b86ed29c8f02d8987`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:41:31.437158 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-1fefedf0-4390-4c29-bfd4-860f9fb196fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:31.513837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1365/2745] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:41:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:41:31.524441 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a3283-2615-4cae-8241-1d1532b4ef17 service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:31.525994 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-212a3283-2615-4cae-8241-1d1532b4ef17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:31.546141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:41:31.586374 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a3283-2615-4cae-8241-1d1532b4ef17 service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:31.591141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52596bd2-46e2-4314-9c7e-b12e22f3895b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:31.591746 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52596bd2-46e2-4314-9c7e-b12e22f3895b service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:31.592016 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52596bd2-46e2-4314-9c7e-b12e22f3895b service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:31.592279 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52596bd2-46e2-4314-9c7e-b12e22f3895b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:31.603331 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52596bd2-46e2-4314-9c7e-b12e22f3895b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:31.604767 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52596bd2-46e2-4314-9c7e-b12e22f3895b service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:31.614051 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1381/2746] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:41:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:41:34.483021 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75ead4-60a6-41a3-9b45-9962db79c6ae None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:34.483584 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75ead4-60a6-41a3-9b45-9962db79c6ae None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:34.483584 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75ead4-60a6-41a3-9b45-9962db79c6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:34.505682 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-4a75ead4-60a6-41a3-9b45-9962db79c6ae None None] MFA Rules not processed for user `4fac6277983e497b86ed29c8f02d8987`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:41:34.506957 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-4a75ead4-60a6-41a3-9b45-9962db79c6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:34.518263 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1366/2747] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:41:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:41:41.665926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=19 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:41:44.498711 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=44 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:41:49.806105 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52596bd2-46e2-4314-9c7e-b12e22f3895b service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:49.809628 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-52596bd2-46e2-4314-9c7e-b12e22f3895b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:49.862103 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52596bd2-46e2-4314-9c7e-b12e22f3895b service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:49.870362 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82deb47-fb30-44c4-940d-65673b6efeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:49.871759 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82deb47-fb30-44c4-940d-65673b6efeb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:49.872255 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82deb47-fb30-44c4-940d-65673b6efeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:49.872772 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82deb47-fb30-44c4-940d-65673b6efeb3 admin admin] PATH_INFO: `/v3/users/f8e574f9176b4946988c47f074bc9e09` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:49.875120 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82deb47-fb30-44c4-940d-65673b6efeb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8e574f9176b4946988c47f074bc9e09)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:49.878877 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82deb47-fb30-44c4-940d-65673b6efeb3 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:50.112819 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c82deb47-fb30-44c4-940d-65673b6efeb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8e574f9176b4946988c47f074bc9e09'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:50.127350 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c82deb47-fb30-44c4-940d-65673b6efeb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8e574f9176b4946988c47f074bc9e09'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:50.136005 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c82deb47-fb30-44c4-940d-65673b6efeb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8e574f9176b4946988c47f074bc9e09'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:50.247439 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c82deb47-fb30-44c4-940d-65673b6efeb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8e574f9176b4946988c47f074bc9e09'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:50.248197 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-c82deb47-fb30-44c4-940d-65673b6efeb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8e574f9176b4946988c47f074bc9e09'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:50.255641 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1382/2748] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:41:49 2022] DELETE /identity/v3/users/f8e574f9176b4946988c47f074bc9e09 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:41:50.262297 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c349c5e-436f-4738-98ab-05c9734ef2c0 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:50.263032 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c349c5e-436f-4738-98ab-05c9734ef2c0 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:50.263462 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c349c5e-436f-4738-98ab-05c9734ef2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:50.292251 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-9c349c5e-436f-4738-98ab-05c9734ef2c0 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:41:50.293594 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-9c349c5e-436f-4738-98ab-05c9734ef2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:50.368811 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1367/2749] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:41:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:41:50.379414 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82deb47-fb30-44c4-940d-65673b6efeb3 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:50.380973 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-c82deb47-fb30-44c4-940d-65673b6efeb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:50.469848 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82deb47-fb30-44c4-940d-65673b6efeb3 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:50.473723 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4721d0f7-0d56-41ac-be87-c617b4e5d10a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:50.474383 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721d0f7-0d56-41ac-be87-c617b4e5d10a service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:50.474691 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721d0f7-0d56-41ac-be87-c617b4e5d10a service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:50.474954 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721d0f7-0d56-41ac-be87-c617b4e5d10a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:50.485703 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4721d0f7-0d56-41ac-be87-c617b4e5d10a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:50.487293 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4721d0f7-0d56-41ac-be87-c617b4e5d10a service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:50.499103 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1383/2750] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:41:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:41:51.074226 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c349c5e-436f-4738-98ab-05c9734ef2c0 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:51.076833 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-9c349c5e-436f-4738-98ab-05c9734ef2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:51.109101 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c349c5e-436f-4738-98ab-05c9734ef2c0 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:51.117305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bff437-0827-4c3b-b35b-30c61be32f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:51.118017 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bff437-0827-4c3b-b35b-30c61be32f8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:51.118378 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bff437-0827-4c3b-b35b-30c61be32f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:51.118737 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bff437-0827-4c3b-b35b-30c61be32f8b admin admin] PATH_INFO: `/v3/projects/bed5e9580fd74fe690c11572487e6525` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:51.120882 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bff437-0827-4c3b-b35b-30c61be32f8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bed5e9580fd74fe690c11572487e6525)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:51.122944 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bff437-0827-4c3b-b35b-30c61be32f8b admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:51.304518 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-63bff437-0827-4c3b-b35b-30c61be32f8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bed5e9580fd74fe690c11572487e6525', 'request_id': 'req-63bff437-0827-4c3b-b35b-30c61be32f8b'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:51.313296 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-63bff437-0827-4c3b-b35b-30c61be32f8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bed5e9580fd74fe690c11572487e6525', 'request_id': 'req-63bff437-0827-4c3b-b35b-30c61be32f8b'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:51.321195 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-63bff437-0827-4c3b-b35b-30c61be32f8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bed5e9580fd74fe690c11572487e6525', 'request_id': 'req-63bff437-0827-4c3b-b35b-30c61be32f8b'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:51.420248 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-63bff437-0827-4c3b-b35b-30c61be32f8b admin admin] The token cache is being invalidate because project bed5e9580fd74fe690c11572487e6525 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:41:51.420546 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-63bff437-0827-4c3b-b35b-30c61be32f8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:51.421222 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-63bff437-0827-4c3b-b35b-30c61be32f8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:41:51.422883 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1368/2751] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:41:51 2022] DELETE /identity/v3/projects/bed5e9580fd74fe690c11572487e6525 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:41:51.428071 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab349aff-1ce9-4235-9cd5-60be849773b2 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:51.428903 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab349aff-1ce9-4235-9cd5-60be849773b2 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:51.429059 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab349aff-1ce9-4235-9cd5-60be849773b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:51.451895 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-ab349aff-1ce9-4235-9cd5-60be849773b2 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:41:51.453867 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-ab349aff-1ce9-4235-9cd5-60be849773b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:51.513363 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:41:51.530751 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1384/2752] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:41:51.539360 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8508d36-07bf-4495-b3ba-cb9f13f99661 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:51.539630 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8508d36-07bf-4495-b3ba-cb9f13f99661 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:51.539895 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8508d36-07bf-4495-b3ba-cb9f13f99661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:51.564507 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-e8508d36-07bf-4495-b3ba-cb9f13f99661 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:41:51.566316 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-e8508d36-07bf-4495-b3ba-cb9f13f99661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:51.575469 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1369/2753] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:41:51.580439 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab349aff-1ce9-4235-9cd5-60be849773b2 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:51.591682 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab349aff-1ce9-4235-9cd5-60be849773b2 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:51.598343 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f331f-01e4-4f53-acda-35a0924e287a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:51.598972 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f331f-01e4-4f53-acda-35a0924e287a admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:51.599266 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f331f-01e4-4f53-acda-35a0924e287a admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:51.599575 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f331f-01e4-4f53-acda-35a0924e287a admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:51.600283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49f331f-01e4-4f53-acda-35a0924e287a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:41:51.601310 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49f331f-01e4-4f53-acda-35a0924e287a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:51.603046 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49f331f-01e4-4f53-acda-35a0924e287a admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:51.623220 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1385/2754] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:41:51.627981 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8508d36-07bf-4495-b3ba-cb9f13f99661 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:51.639782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8508d36-07bf-4495-b3ba-cb9f13f99661 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:51.645766 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a28a71-eab0-458c-a0a7-8ef10591244f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:51.646583 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a28a71-eab0-458c-a0a7-8ef10591244f admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:51.647116 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a28a71-eab0-458c-a0a7-8ef10591244f admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:51.647535 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a28a71-eab0-458c-a0a7-8ef10591244f admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:51.648602 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a28a71-eab0-458c-a0a7-8ef10591244f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:51.652815 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a28a71-eab0-458c-a0a7-8ef10591244f admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:51.765507 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1370/2755] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:41:51 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:41:51.770190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f331f-01e4-4f53-acda-35a0924e287a admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:51.783491 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f331f-01e4-4f53-acda-35a0924e287a admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:51.851485 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7206ce-a85c-4f57-9569-c8b5d84ba345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:51.852068 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7206ce-a85c-4f57-9569-c8b5d84ba345 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:51.852428 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7206ce-a85c-4f57-9569-c8b5d84ba345 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:51.852699 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7206ce-a85c-4f57-9569-c8b5d84ba345 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:51.853348 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7206ce-a85c-4f57-9569-c8b5d84ba345 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:51.855418 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7206ce-a85c-4f57-9569-c8b5d84ba345 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:52.019808 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1386/2756] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:41:51 2022] POST /identity/v3/users => generated 288 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:41:52.024288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a28a71-eab0-458c-a0a7-8ef10591244f admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:52.036308 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a28a71-eab0-458c-a0a7-8ef10591244f admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:52.041502 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969396f3-e4be-44cf-bfa8-ffa3ca8200fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:52.042132 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969396f3-e4be-44cf-bfa8-ffa3ca8200fa admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:52.042439 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969396f3-e4be-44cf-bfa8-ffa3ca8200fa admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:52.042714 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969396f3-e4be-44cf-bfa8-ffa3ca8200fa admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:52.043392 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969396f3-e4be-44cf-bfa8-ffa3ca8200fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:52.045058 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969396f3-e4be-44cf-bfa8-ffa3ca8200fa admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:52.056542 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1371/2757] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:41:52.061470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7206ce-a85c-4f57-9569-c8b5d84ba345 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:52.072583 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7206ce-a85c-4f57-9569-c8b5d84ba345 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:52.079207 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e691f-71c0-4252-b204-ad8c9379378f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:52.079814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e691f-71c0-4252-b204-ad8c9379378f admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:52.080153 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e691f-71c0-4252-b204-ad8c9379378f admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:52.080465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e691f-71c0-4252-b204-ad8c9379378f admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:52.081088 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e691f-71c0-4252-b204-ad8c9379378f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:52.082753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e691f-71c0-4252-b204-ad8c9379378f admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:52.098333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1387/2758] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:41:52.103328 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969396f3-e4be-44cf-bfa8-ffa3ca8200fa admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:52.117776 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969396f3-e4be-44cf-bfa8-ffa3ca8200fa admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:52.123887 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b4496-b691-47ba-8702-b9e83bb96c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:52.124792 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b4496-b691-47ba-8702-b9e83bb96c13 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:52.126826 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b4496-b691-47ba-8702-b9e83bb96c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:52.126826 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b4496-b691-47ba-8702-b9e83bb96c13 admin admin] PATH_INFO: `/v3/projects/3a14d4c1e3504685ba0dc1309595dab7/users/76bad0895fc446d084fec6494077eda0/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:52.146066 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b4496-b691-47ba-8702-b9e83bb96c13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a14d4c1e3504685ba0dc1309595dab7, user_id=76bad0895fc446d084fec6494077eda0, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:52.149034 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b4496-b691-47ba-8702-b9e83bb96c13 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:52.258924 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1372/2759] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:41:52 2022] PUT /identity/v3/projects/3a14d4c1e3504685ba0dc1309595dab7/users/76bad0895fc446d084fec6494077eda0/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:41:52.281428 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a99f8-9183-4b67-a59d-fa176d4e5c73 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:52.281964 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a99f8-9183-4b67-a59d-fa176d4e5c73 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:52.282446 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a99f8-9183-4b67-a59d-fa176d4e5c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:52.316002 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-8e4a99f8-9183-4b67-a59d-fa176d4e5c73 None None] MFA Rules not processed for user `76bad0895fc446d084fec6494077eda0`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:41:52.317336 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-8e4a99f8-9183-4b67-a59d-fa176d4e5c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:52.432920 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1388/2760] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:41:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:41:52.444715 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b4496-b691-47ba-8702-b9e83bb96c13 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:52.447009 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-dc2b4496-b691-47ba-8702-b9e83bb96c13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:52.473437 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:41:52.526552 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b4496-b691-47ba-8702-b9e83bb96c13 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:52.530903 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc2716-a8b1-4d93-997e-84a74f2c3767 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:52.531689 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc2716-a8b1-4d93-997e-84a74f2c3767 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:52.532009 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc2716-a8b1-4d93-997e-84a74f2c3767 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:52.532345 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc2716-a8b1-4d93-997e-84a74f2c3767 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:52.542902 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dc2716-a8b1-4d93-997e-84a74f2c3767 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:52.544650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dc2716-a8b1-4d93-997e-84a74f2c3767 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:52.555530 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1373/2761] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:41:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:41:55.583338 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a99f8-9183-4b67-a59d-fa176d4e5c73 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:41:55.585146 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-8e4a99f8-9183-4b67-a59d-fa176d4e5c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:55.611906 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4b067-bba4-4b4f-b522-834fba68dc03 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:55.612604 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4b067-bba4-4b4f-b522-834fba68dc03 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:55.612604 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4b067-bba4-4b4f-b522-834fba68dc03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:55.633805 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-65a4b067-bba4-4b4f-b522-834fba68dc03 None None] MFA Rules not processed for user `76bad0895fc446d084fec6494077eda0`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:41:55.636338 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-65a4b067-bba4-4b4f-b522-834fba68dc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:55.645018 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1374/2762] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:41:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:41:55.665516 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:41:55.722982 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a99f8-9183-4b67-a59d-fa176d4e5c73 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:41:55.731431 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967bdaac-cb65-4a25-8417-1a1bf1f0ef82 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3db46a4b4e74cc0bb2a7bd6f0e99105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:41:55.732080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967bdaac-cb65-4a25-8417-1a1bf1f0ef82 service placement] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:55.732840 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967bdaac-cb65-4a25-8417-1a1bf1f0ef82 service placement] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:55.733116 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967bdaac-cb65-4a25-8417-1a1bf1f0ef82 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:55.736580 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-967bdaac-cb65-4a25-8417-1a1bf1f0ef82 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:55.759281 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967bdaac-cb65-4a25-8417-1a1bf1f0ef82 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:41:55.762289 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967bdaac-cb65-4a25-8417-1a1bf1f0ef82 service placement] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:41:55.778132 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1389/2763] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:41:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:41:57.396815 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c15454-2e1a-4e3d-b2f8-e062b99bb784 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:41:57.397374 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c15454-2e1a-4e3d-b2f8-e062b99bb784 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:41:57.397374 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c15454-2e1a-4e3d-b2f8-e062b99bb784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:41:57.420788 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-b9c15454-2e1a-4e3d-b2f8-e062b99bb784 None None] MFA Rules not processed for user `4fac6277983e497b86ed29c8f02d8987`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:41:57.423065 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-b9c15454-2e1a-4e3d-b2f8-e062b99bb784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:41:57.533279 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1375/2764] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:41:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:42:05.770529 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=20 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:42:07.526837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=45 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:42:18.528202 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09dc1d9-c970-4473-a8bc-b2f9d4681760 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:18.528757 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09dc1d9-c970-4473-a8bc-b2f9d4681760 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:18.528757 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09dc1d9-c970-4473-a8bc-b2f9d4681760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:18.601020 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-a09dc1d9-c970-4473-a8bc-b2f9d4681760 None None] MFA Rules not processed for user `7796ebff70054de090e02c58bf7000f4`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:18.602906 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a09dc1d9-c970-4473-a8bc-b2f9d4681760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:18.692237 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1390/2765] 10.222.0.16 () {58 vars in 1009 bytes} [Wed Aug 31 11:42:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:42:18.856979 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c15454-2e1a-4e3d-b2f8-e062b99bb784 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:18.858632 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-b9c15454-2e1a-4e3d-b2f8-e062b99bb784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:18.959010 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c15454-2e1a-4e3d-b2f8-e062b99bb784 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:18.965141 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257ee5c-cc1b-4607-805e-829675b410fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8abc7c439a04622896b34ed0a7b8492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:18.965803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257ee5c-cc1b-4607-805e-829675b410fe service swift] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:18.966083 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257ee5c-cc1b-4607-805e-829675b410fe service swift] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:18.966385 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257ee5c-cc1b-4607-805e-829675b410fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:18.977611 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257ee5c-cc1b-4607-805e-829675b410fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:18.979302 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257ee5c-cc1b-4607-805e-829675b410fe service swift] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:18.988616 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1376/2766] 10.222.0.16 () {60 vars in 1384 bytes} [Wed Aug 31 11:42:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:42:20.120996 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06606c-c827-4558-8b64-c470bb778742 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:20.120996 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06606c-c827-4558-8b64-c470bb778742 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:20.120996 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06606c-c827-4558-8b64-c470bb778742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:20.143271 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-bc06606c-c827-4558-8b64-c470bb778742 None None] MFA Rules not processed for user `7796ebff70054de090e02c58bf7000f4`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:20.144524 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-bc06606c-c827-4558-8b64-c470bb778742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:20.153182 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1391/2767] 10.222.0.16 () {58 vars in 967 bytes} [Wed Aug 31 11:42:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:20.168502 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257ee5c-cc1b-4607-805e-829675b410fe service swift] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:20.179469 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257ee5c-cc1b-4607-805e-829675b410fe service swift] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:20.185935 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12b26a-192d-425e-8065-f8224b60061e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8abc7c439a04622896b34ed0a7b8492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:20.186600 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12b26a-192d-425e-8065-f8224b60061e service swift] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:20.186878 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12b26a-192d-425e-8065-f8224b60061e service swift] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:20.187376 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12b26a-192d-425e-8065-f8224b60061e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:20.198419 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12b26a-192d-425e-8065-f8224b60061e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:20.207176 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12b26a-192d-425e-8065-f8224b60061e service swift] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:20.217043 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1377/2768] 10.222.0.16 () {60 vars in 1384 bytes} [Wed Aug 31 11:42:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:42:20.938552 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62055791-7218-4453-b06b-4ed1faa37f5b None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:20.938835 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62055791-7218-4453-b06b-4ed1faa37f5b None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:20.939104 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62055791-7218-4453-b06b-4ed1faa37f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:20.960152 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-62055791-7218-4453-b06b-4ed1faa37f5b None None] MFA Rules not processed for user `7796ebff70054de090e02c58bf7000f4`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:20.961981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-62055791-7218-4453-b06b-4ed1faa37f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:20.969964 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1392/2769] 10.222.0.16 () {58 vars in 1009 bytes} [Wed Aug 31 11:42:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:20.982630 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12b26a-192d-425e-8065-f8224b60061e service swift] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:20.995051 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12b26a-192d-425e-8065-f8224b60061e service swift] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:21.000678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e155d7-3cb8-4c6e-be58-3d1d9ba4b9cd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8abc7c439a04622896b34ed0a7b8492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:21.001457 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e155d7-3cb8-4c6e-be58-3d1d9ba4b9cd service swift] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:21.001776 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e155d7-3cb8-4c6e-be58-3d1d9ba4b9cd service swift] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:21.002087 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e155d7-3cb8-4c6e-be58-3d1d9ba4b9cd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:21.012388 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e155d7-3cb8-4c6e-be58-3d1d9ba4b9cd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:21.013984 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e155d7-3cb8-4c6e-be58-3d1d9ba4b9cd service swift] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:21.023901 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1378/2770] 10.222.0.16 () {60 vars in 1384 bytes} [Wed Aug 31 11:42:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:42:21.829054 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623ddcf-2271-4668-b6c5-d71661ed50b9 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:21.829890 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623ddcf-2271-4668-b6c5-d71661ed50b9 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:21.829890 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623ddcf-2271-4668-b6c5-d71661ed50b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:21.853953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-3623ddcf-2271-4668-b6c5-d71661ed50b9 None None] MFA Rules not processed for user `7796ebff70054de090e02c58bf7000f4`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:21.855260 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-3623ddcf-2271-4668-b6c5-d71661ed50b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:21.864203 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1393/2771] 10.222.0.16 () {58 vars in 967 bytes} [Wed Aug 31 11:42:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:42:21.877931 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e155d7-3cb8-4c6e-be58-3d1d9ba4b9cd service swift] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:21.890622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e155d7-3cb8-4c6e-be58-3d1d9ba4b9cd service swift] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:21.899600 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4da908-f0bc-4d18-abfa-4b26b6412cc6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8abc7c439a04622896b34ed0a7b8492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:21.901076 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4da908-f0bc-4d18-abfa-4b26b6412cc6 service swift] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:21.901076 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4da908-f0bc-4d18-abfa-4b26b6412cc6 service swift] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:21.901375 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4da908-f0bc-4d18-abfa-4b26b6412cc6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:21.912025 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4da908-f0bc-4d18-abfa-4b26b6412cc6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:21.914041 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4da908-f0bc-4d18-abfa-4b26b6412cc6 service swift] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:21.926615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1379/2772] 10.222.0.16 () {60 vars in 1384 bytes} [Wed Aug 31 11:42:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 11:42:25.323059 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623ddcf-2271-4668-b6c5-d71661ed50b9 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:25.421553 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623ddcf-2271-4668-b6c5-d71661ed50b9 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:25.427188 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c7669-55eb-4eec-ab79-765696336289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:25.428083 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c7669-55eb-4eec-ab79-765696336289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:25.428436 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c7669-55eb-4eec-ab79-765696336289 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:25.428787 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c7669-55eb-4eec-ab79-765696336289 admin admin] PATH_INFO: `/v3/users/76bad0895fc446d084fec6494077eda0` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:25.431786 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259c7669-55eb-4eec-ab79-765696336289 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76bad0895fc446d084fec6494077eda0)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:25.434379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259c7669-55eb-4eec-ab79-765696336289 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:25.619047 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:42:25.621245 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-259c7669-55eb-4eec-ab79-765696336289 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76bad0895fc446d084fec6494077eda0'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:42:25.650814 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-259c7669-55eb-4eec-ab79-765696336289 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76bad0895fc446d084fec6494077eda0'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:42:25.669629 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-259c7669-55eb-4eec-ab79-765696336289 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76bad0895fc446d084fec6494077eda0'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:42:25.794779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-259c7669-55eb-4eec-ab79-765696336289 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76bad0895fc446d084fec6494077eda0'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:42:25.795947 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-259c7669-55eb-4eec-ab79-765696336289 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76bad0895fc446d084fec6494077eda0'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:42:25.803633 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1394/2773] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:42:25 2022] DELETE /identity/v3/users/76bad0895fc446d084fec6494077eda0 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:42:25.811402 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9b0d7-bda3-40c7-82ba-7c82b4e89240 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:25.812090 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9b0d7-bda3-40c7-82ba-7c82b4e89240 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:25.812419 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9b0d7-bda3-40c7-82ba-7c82b4e89240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:25.852320 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-ffd9b0d7-bda3-40c7-82ba-7c82b4e89240 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:25.853720 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-ffd9b0d7-bda3-40c7-82ba-7c82b4e89240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:25.925402 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1380/2774] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:42:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:25.937377 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c7669-55eb-4eec-ab79-765696336289 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:25.939177 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-259c7669-55eb-4eec-ab79-765696336289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:26.036984 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c7669-55eb-4eec-ab79-765696336289 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:26.044602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6bffca-567c-4dfd-93cd-9983b9288f01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:26.045680 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bffca-567c-4dfd-93cd-9983b9288f01 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:26.046144 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bffca-567c-4dfd-93cd-9983b9288f01 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:26.046689 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bffca-567c-4dfd-93cd-9983b9288f01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:26.061993 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6bffca-567c-4dfd-93cd-9983b9288f01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:26.064088 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6bffca-567c-4dfd-93cd-9983b9288f01 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:26.077471 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1395/2775] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:42:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:42:26.428141 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9b0d7-bda3-40c7-82ba-7c82b4e89240 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:26.428141 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-ffd9b0d7-bda3-40c7-82ba-7c82b4e89240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:26.462204 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9b0d7-bda3-40c7-82ba-7c82b4e89240 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:26.468822 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a3588-9870-476a-b7a6-0998c75eb995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:26.469459 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a3588-9870-476a-b7a6-0998c75eb995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:26.469934 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a3588-9870-476a-b7a6-0998c75eb995 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:26.470360 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a3588-9870-476a-b7a6-0998c75eb995 admin admin] PATH_INFO: `/v3/projects/3a14d4c1e3504685ba0dc1309595dab7` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:26.474494 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a3588-9870-476a-b7a6-0998c75eb995 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a14d4c1e3504685ba0dc1309595dab7)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:26.477176 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a3588-9870-476a-b7a6-0998c75eb995 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:26.595968 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:42:26.615570 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5e6a3588-9870-476a-b7a6-0998c75eb995 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a14d4c1e3504685ba0dc1309595dab7', 'request_id': 'req-5e6a3588-9870-476a-b7a6-0998c75eb995'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:42:26.625633 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5e6a3588-9870-476a-b7a6-0998c75eb995 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a14d4c1e3504685ba0dc1309595dab7', 'request_id': 'req-5e6a3588-9870-476a-b7a6-0998c75eb995'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:42:26.632924 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5e6a3588-9870-476a-b7a6-0998c75eb995 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a14d4c1e3504685ba0dc1309595dab7', 'request_id': 'req-5e6a3588-9870-476a-b7a6-0998c75eb995'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:42:26.753892 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5e6a3588-9870-476a-b7a6-0998c75eb995 admin admin] The token cache is being invalidate because project 3a14d4c1e3504685ba0dc1309595dab7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:42:26.754139 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5e6a3588-9870-476a-b7a6-0998c75eb995 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:42:26.754804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-5e6a3588-9870-476a-b7a6-0998c75eb995 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:42:26.756366 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1381/2776] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:42:26 2022] DELETE /identity/v3/projects/3a14d4c1e3504685ba0dc1309595dab7 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:42:26.762643 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f50f6-d494-4d9b-bf19-15bdf16f92e1 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:26.763555 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f50f6-d494-4d9b-bf19-15bdf16f92e1 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:26.763555 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f50f6-d494-4d9b-bf19-15bdf16f92e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:26.785912 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-347f50f6-d494-4d9b-bf19-15bdf16f92e1 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:26.787193 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-347f50f6-d494-4d9b-bf19-15bdf16f92e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:26.859336 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1396/2777] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:42:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:26.870097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd53559-f943-4488-ac36-ff4f6daa3717 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:26.870672 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd53559-f943-4488-ac36-ff4f6daa3717 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:26.870672 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd53559-f943-4488-ac36-ff4f6daa3717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:26.893666 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-0cd53559-f943-4488-ac36-ff4f6daa3717 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:26.895057 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-0cd53559-f943-4488-ac36-ff4f6daa3717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:26.904145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1382/2778] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:42:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:26.908984 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f50f6-d494-4d9b-bf19-15bdf16f92e1 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:26.921544 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f50f6-d494-4d9b-bf19-15bdf16f92e1 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:26.928320 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85cf69-3661-4e4c-a0e9-c829c4a413da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:26.929148 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85cf69-3661-4e4c-a0e9-c829c4a413da admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:26.929586 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85cf69-3661-4e4c-a0e9-c829c4a413da admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:26.929959 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85cf69-3661-4e4c-a0e9-c829c4a413da admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:26.931143 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85cf69-3661-4e4c-a0e9-c829c4a413da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:42:26.931560 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85cf69-3661-4e4c-a0e9-c829c4a413da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:26.933388 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85cf69-3661-4e4c-a0e9-c829c4a413da admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:26.949938 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1397/2779] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:42:26.957438 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd53559-f943-4488-ac36-ff4f6daa3717 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:26.975181 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd53559-f943-4488-ac36-ff4f6daa3717 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:26.983759 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58c920-528b-456d-8080-f2bb5d4af5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:26.984845 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58c920-528b-456d-8080-f2bb5d4af5dc admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:26.986487 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58c920-528b-456d-8080-f2bb5d4af5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:26.986487 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58c920-528b-456d-8080-f2bb5d4af5dc admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:26.987038 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e58c920-528b-456d-8080-f2bb5d4af5dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:26.990716 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e58c920-528b-456d-8080-f2bb5d4af5dc admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:27.169268 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1383/2780] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:42:26 2022] POST /identity/v3/projects => generated 377 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:42:27.173978 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85cf69-3661-4e4c-a0e9-c829c4a413da admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:27.186962 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85cf69-3661-4e4c-a0e9-c829c4a413da admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:27.252609 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:42:27.262205 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce39caf-7c93-4546-bf4a-e36647612e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:27.262861 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce39caf-7c93-4546-bf4a-e36647612e68 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:27.263141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce39caf-7c93-4546-bf4a-e36647612e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:27.263435 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce39caf-7c93-4546-bf4a-e36647612e68 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:27.264084 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce39caf-7c93-4546-bf4a-e36647612e68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:27.266077 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce39caf-7c93-4546-bf4a-e36647612e68 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:27.406574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1398/2781] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:42:27 2022] POST /identity/v3/users => generated 289 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:42:27.412695 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58c920-528b-456d-8080-f2bb5d4af5dc admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:27.424514 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58c920-528b-456d-8080-f2bb5d4af5dc admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:27.430108 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759cd67-8c50-4227-ab65-1e7af750b122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:27.430785 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759cd67-8c50-4227-ab65-1e7af750b122 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:27.431168 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759cd67-8c50-4227-ab65-1e7af750b122 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:27.431444 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759cd67-8c50-4227-ab65-1e7af750b122 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:27.432109 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759cd67-8c50-4227-ab65-1e7af750b122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:27.433831 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759cd67-8c50-4227-ab65-1e7af750b122 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:27.446156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1384/2782] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:42:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:42:27.457000 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce39caf-7c93-4546-bf4a-e36647612e68 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:27.472551 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce39caf-7c93-4546-bf4a-e36647612e68 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:27.481903 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82e929-7589-495e-a302-fedac58f6124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:27.488650 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82e929-7589-495e-a302-fedac58f6124 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:27.488650 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82e929-7589-495e-a302-fedac58f6124 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:27.488650 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82e929-7589-495e-a302-fedac58f6124 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:27.488650 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef82e929-7589-495e-a302-fedac58f6124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:27.488940 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef82e929-7589-495e-a302-fedac58f6124 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:27.510172 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1399/2783] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:42:27 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:42:27.514295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759cd67-8c50-4227-ab65-1e7af750b122 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:27.528059 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759cd67-8c50-4227-ab65-1e7af750b122 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:27.535783 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377619a-fb0e-4f15-973c-a2e9dce53601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:27.536899 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377619a-fb0e-4f15-973c-a2e9dce53601 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:27.537737 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377619a-fb0e-4f15-973c-a2e9dce53601 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:27.538197 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377619a-fb0e-4f15-973c-a2e9dce53601 admin admin] PATH_INFO: `/v3/projects/09c0f45a215341de9ab189e544a6ecd6/users/8492a5ac560f43e4a3342f067dda8fef/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:27.561883 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d377619a-fb0e-4f15-973c-a2e9dce53601 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09c0f45a215341de9ab189e544a6ecd6, user_id=8492a5ac560f43e4a3342f067dda8fef, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:27.564936 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d377619a-fb0e-4f15-973c-a2e9dce53601 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:27.653952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1385/2784] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:42:27 2022] PUT /identity/v3/projects/09c0f45a215341de9ab189e544a6ecd6/users/8492a5ac560f43e4a3342f067dda8fef/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:42:27.660238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3d452-9bc4-4934-939f-cf17a136a32a None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:27.660574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3d452-9bc4-4934-939f-cf17a136a32a None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:27.660888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3d452-9bc4-4934-939f-cf17a136a32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:27.684126 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-a2a3d452-9bc4-4934-939f-cf17a136a32a None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:27.686181 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a2a3d452-9bc4-4934-939f-cf17a136a32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:27.763030 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1400/2785] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:42:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:42:27.774860 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377619a-fb0e-4f15-973c-a2e9dce53601 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:27.776528 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-d377619a-fb0e-4f15-973c-a2e9dce53601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:27.845930 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377619a-fb0e-4f15-973c-a2e9dce53601 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:27.850318 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb78b9-393b-4a1a-aee4-c13e67147c21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:27.850995 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb78b9-393b-4a1a-aee4-c13e67147c21 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:27.851273 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb78b9-393b-4a1a-aee4-c13e67147c21 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:27.851758 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb78b9-393b-4a1a-aee4-c13e67147c21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:27.863368 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb78b9-393b-4a1a-aee4-c13e67147c21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:27.864912 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb78b9-393b-4a1a-aee4-c13e67147c21 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:27.876544 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1386/2786] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:42:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:42:33.552668 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91313e36-5ff0-4f41-959b-83130d3350f1 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:33.553566 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91313e36-5ff0-4f41-959b-83130d3350f1 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:33.553827 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91313e36-5ff0-4f41-959b-83130d3350f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:33.593223 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-91313e36-5ff0-4f41-959b-83130d3350f1 None None] MFA Rules not processed for user `7796ebff70054de090e02c58bf7000f4`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:33.593223 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-91313e36-5ff0-4f41-959b-83130d3350f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:33.668034 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1401/2787] 10.222.0.16 () {58 vars in 1009 bytes} [Wed Aug 31 11:42:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:33.682531 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb78b9-393b-4a1a-aee4-c13e67147c21 service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:33.684773 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-21fb78b9-393b-4a1a-aee4-c13e67147c21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:33.754641 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:42:33.780909 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb78b9-393b-4a1a-aee4-c13e67147c21 service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:33.784855 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a17fe4-ac33-4087-b09f-4f7e9ec5d6ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8abc7c439a04622896b34ed0a7b8492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:33.785642 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a17fe4-ac33-4087-b09f-4f7e9ec5d6ed service swift] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:33.785987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a17fe4-ac33-4087-b09f-4f7e9ec5d6ed service swift] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:33.786436 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a17fe4-ac33-4087-b09f-4f7e9ec5d6ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:33.796722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a17fe4-ac33-4087-b09f-4f7e9ec5d6ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:33.798541 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a17fe4-ac33-4087-b09f-4f7e9ec5d6ed service swift] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:33.809427 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1387/2788] 10.222.0.16 () {60 vars in 1384 bytes} [Wed Aug 31 11:42:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:42:35.899763 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd9092-a6f3-4638-9f05-9543a2ae81b5 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:35.900915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd9092-a6f3-4638-9f05-9543a2ae81b5 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:35.900915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd9092-a6f3-4638-9f05-9543a2ae81b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:35.967020 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-32cd9092-a6f3-4638-9f05-9543a2ae81b5 None None] MFA Rules not processed for user `8492a5ac560f43e4a3342f067dda8fef`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:35.969999 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-32cd9092-a6f3-4638-9f05-9543a2ae81b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:36.085951 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1402/2789] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:42:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:36.091345 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a17fe4-ac33-4087-b09f-4f7e9ec5d6ed service swift] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:36.122811 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a17fe4-ac33-4087-b09f-4f7e9ec5d6ed service swift] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:36.129238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6180baa6-4ea1-4b0c-a77c-d874ff796ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:36.130114 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180baa6-4ea1-4b0c-a77c-d874ff796ac6 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:36.130491 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180baa6-4ea1-4b0c-a77c-d874ff796ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:36.130837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180baa6-4ea1-4b0c-a77c-d874ff796ac6 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:36.131628 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6180baa6-4ea1-4b0c-a77c-d874ff796ac6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:36.134753 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6180baa6-4ea1-4b0c-a77c-d874ff796ac6 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:36.262549 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1388/2790] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:42:36 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:42:36.266681 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd9092-a6f3-4638-9f05-9543a2ae81b5 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:36.285405 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd9092-a6f3-4638-9f05-9543a2ae81b5 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:36.293001 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:42:36.413630 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b9478-9ca4-4e3a-9c86-b9ff675b42ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:36.414759 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b9478-9ca4-4e3a-9c86-b9ff675b42ac admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:36.415030 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b9478-9ca4-4e3a-9c86-b9ff675b42ac admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:36.415302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b9478-9ca4-4e3a-9c86-b9ff675b42ac admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:36.415974 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b9478-9ca4-4e3a-9c86-b9ff675b42ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:36.417969 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b9478-9ca4-4e3a-9c86-b9ff675b42ac admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:36.522881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1403/2791] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:42:36 2022] POST /identity/v3/users => generated 295 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:42:36.529854 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6180baa6-4ea1-4b0c-a77c-d874ff796ac6 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:36.555504 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6180baa6-4ea1-4b0c-a77c-d874ff796ac6 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:36.567200 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8fdaa2-0e85-421d-8fba-a206276f5144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:36.568239 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fdaa2-0e85-421d-8fba-a206276f5144 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:36.568869 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fdaa2-0e85-421d-8fba-a206276f5144 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:36.569385 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fdaa2-0e85-421d-8fba-a206276f5144 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:36.571013 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8fdaa2-0e85-421d-8fba-a206276f5144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:36.573067 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8fdaa2-0e85-421d-8fba-a206276f5144 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:36.590717 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1389/2792] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:42:36.595978 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b9478-9ca4-4e3a-9c86-b9ff675b42ac admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:36.618041 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b9478-9ca4-4e3a-9c86-b9ff675b42ac admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:36.628735 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd451e-aed9-4e64-92a1-94745c3616e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:36.628735 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd451e-aed9-4e64-92a1-94745c3616e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:36.628735 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd451e-aed9-4e64-92a1-94745c3616e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:36.628735 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd451e-aed9-4e64-92a1-94745c3616e8 admin admin] PATH_INFO: `/v3/projects/98ee52a6fe054db199b8f08bd5b6052e/users/646f1e7724f14f29bf806bcc3a6a554c/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:36.650939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd451e-aed9-4e64-92a1-94745c3616e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98ee52a6fe054db199b8f08bd5b6052e, user_id=646f1e7724f14f29bf806bcc3a6a554c, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:36.653917 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd451e-aed9-4e64-92a1-94745c3616e8 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:36.728705 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1404/2793] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:42:36 2022] PUT /identity/v3/projects/98ee52a6fe054db199b8f08bd5b6052e/users/646f1e7724f14f29bf806bcc3a6a554c/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:42:36.733137 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8fdaa2-0e85-421d-8fba-a206276f5144 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:36.749519 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8fdaa2-0e85-421d-8fba-a206276f5144 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:36.877062 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171da3a-f623-4a2a-b4ab-895cba0ce202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:36.883782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171da3a-f623-4a2a-b4ab-895cba0ce202 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:36.883782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171da3a-f623-4a2a-b4ab-895cba0ce202 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:36.883782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171da3a-f623-4a2a-b4ab-895cba0ce202 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:36.883782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7171da3a-f623-4a2a-b4ab-895cba0ce202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:36.883782 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7171da3a-f623-4a2a-b4ab-895cba0ce202 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:36.907323 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1390/2794] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:42:36.908363 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd451e-aed9-4e64-92a1-94745c3616e8 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:36.921056 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd451e-aed9-4e64-92a1-94745c3616e8 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:36.927791 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4054067-bde2-4574-825b-be56542afb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:36.928887 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4054067-bde2-4574-825b-be56542afb74 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:36.929368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4054067-bde2-4574-825b-be56542afb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:36.929635 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4054067-bde2-4574-825b-be56542afb74 admin admin] PATH_INFO: `/v3/projects/98ee52a6fe054db199b8f08bd5b6052e/users/646f1e7724f14f29bf806bcc3a6a554c/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:36.932735 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4054067-bde2-4574-825b-be56542afb74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98ee52a6fe054db199b8f08bd5b6052e, user_id=646f1e7724f14f29bf806bcc3a6a554c, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:36.939516 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4054067-bde2-4574-825b-be56542afb74 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:36.975873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1405/2795] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:42:36 2022] PUT /identity/v3/projects/98ee52a6fe054db199b8f08bd5b6052e/users/646f1e7724f14f29bf806bcc3a6a554c/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:42:45.225584 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171da3a-f623-4a2a-b4ab-895cba0ce202 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:45.331829 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171da3a-f623-4a2a-b4ab-895cba0ce202 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:45.419123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ffac7-a10a-4895-a62d-c5a0040a5920 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:45.419123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ffac7-a10a-4895-a62d-c5a0040a5920 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:45.419123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ffac7-a10a-4895-a62d-c5a0040a5920 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:45.419123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ffac7-a10a-4895-a62d-c5a0040a5920 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:45.421078 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-781ffac7-a10a-4895-a62d-c5a0040a5920 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:45.442740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:42:45.576162 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781ffac7-a10a-4895-a62d-c5a0040a5920 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:45.578512 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781ffac7-a10a-4895-a62d-c5a0040a5920 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:45.594572 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1391/2796] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:42:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:42:46.961478 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=41,keystone:INSERT=5 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:42:48.080537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a6141-b16c-4b7f-bae8-483064d4865e None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:48.081512 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a6141-b16c-4b7f-bae8-483064d4865e None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:48.081512 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a6141-b16c-4b7f-bae8-483064d4865e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:48.139707 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-d85a6141-b16c-4b7f-bae8-483064d4865e None None] MFA Rules not processed for user `646f1e7724f14f29bf806bcc3a6a554c`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:48.149062 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-d85a6141-b16c-4b7f-bae8-483064d4865e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:48.293537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1406/2797] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:42:48 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:42:48.317865 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ffac7-a10a-4895-a62d-c5a0040a5920 service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:42:48.336946 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-781ffac7-a10a-4895-a62d-c5a0040a5920 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:42:48.363133 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ffac7-a10a-4895-a62d-c5a0040a5920 service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:42:48.376279 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac2b35-c3db-4725-81cc-050de1660f76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:42:48.377520 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac2b35-c3db-4725-81cc-050de1660f76 service nova] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:48.378000 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac2b35-c3db-4725-81cc-050de1660f76 service nova] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:48.378493 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac2b35-c3db-4725-81cc-050de1660f76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:48.651041 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac2b35-c3db-4725-81cc-050de1660f76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:42:48.654149 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac2b35-c3db-4725-81cc-050de1660f76 service nova] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:42:48.680487 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1392/2798] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:42:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:42:58.282011 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=34 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:42:58.662891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=54 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:42:59.910384 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842023c-76bc-4b02-999b-0bcd4454327e None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:42:59.911436 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842023c-76bc-4b02-999b-0bcd4454327e None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:42:59.913602 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842023c-76bc-4b02-999b-0bcd4454327e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:42:59.941839 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-b842023c-76bc-4b02-999b-0bcd4454327e None None] MFA Rules not processed for user `7796ebff70054de090e02c58bf7000f4`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:42:59.944286 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-b842023c-76bc-4b02-999b-0bcd4454327e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:00.013109 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1407/2799] 10.222.0.16 () {58 vars in 967 bytes} [Wed Aug 31 11:42:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:00.027793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac2b35-c3db-4725-81cc-050de1660f76 service nova] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:00.210943 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac2b35-c3db-4725-81cc-050de1660f76 service nova] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:00.225507 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152deb58-b692-4386-a21b-ee253ee51967 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8abc7c439a04622896b34ed0a7b8492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:00.231102 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152deb58-b692-4386-a21b-ee253ee51967 service swift] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:00.231102 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152deb58-b692-4386-a21b-ee253ee51967 service swift] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:00.231102 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152deb58-b692-4386-a21b-ee253ee51967 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:00.247045 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152deb58-b692-4386-a21b-ee253ee51967 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:00.250236 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152deb58-b692-4386-a21b-ee253ee51967 service swift] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:00.266560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1393/2800] 10.222.0.16 () {60 vars in 1384 bytes} [Wed Aug 31 11:43:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:43:03.843690 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b842023c-76bc-4b02-999b-0bcd4454327e None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:03.845569 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-b842023c-76bc-4b02-999b-0bcd4454327e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:03.947682 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b842023c-76bc-4b02-999b-0bcd4454327e None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:03.955888 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a8c9f-0672-4999-b7de-3309ffce84f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:03.956957 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a8c9f-0672-4999-b7de-3309ffce84f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:03.957435 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a8c9f-0672-4999-b7de-3309ffce84f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:03.957873 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a8c9f-0672-4999-b7de-3309ffce84f4 admin admin] PATH_INFO: `/v3/users/4fac6277983e497b86ed29c8f02d8987` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:03.960656 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410a8c9f-0672-4999-b7de-3309ffce84f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fac6277983e497b86ed29c8f02d8987)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:03.962781 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410a8c9f-0672-4999-b7de-3309ffce84f4 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:04.228486 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-410a8c9f-0672-4999-b7de-3309ffce84f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fac6277983e497b86ed29c8f02d8987'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:04.243289 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-410a8c9f-0672-4999-b7de-3309ffce84f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fac6277983e497b86ed29c8f02d8987'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:04.252700 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-410a8c9f-0672-4999-b7de-3309ffce84f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fac6277983e497b86ed29c8f02d8987'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:04.340939 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-410a8c9f-0672-4999-b7de-3309ffce84f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fac6277983e497b86ed29c8f02d8987'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:04.341672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-410a8c9f-0672-4999-b7de-3309ffce84f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fac6277983e497b86ed29c8f02d8987'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:04.349433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1408/2801] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:43:03 2022] DELETE /identity/v3/users/4fac6277983e497b86ed29c8f02d8987 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:04.637935 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152deb58-b692-4386-a21b-ee253ee51967 service swift] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:04.653620 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152deb58-b692-4386-a21b-ee253ee51967 service swift] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:04.758265 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91bc308-5174-4040-b457-db3c33f9deb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:04.759626 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bc308-5174-4040-b457-db3c33f9deb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:04.760125 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bc308-5174-4040-b457-db3c33f9deb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:04.760618 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bc308-5174-4040-b457-db3c33f9deb2 admin admin] PATH_INFO: `/v3/projects/a63507e4eb9f4dafbbb6aa5af971632c` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:04.762843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91bc308-5174-4040-b457-db3c33f9deb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a63507e4eb9f4dafbbb6aa5af971632c)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:04.766628 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91bc308-5174-4040-b457-db3c33f9deb2 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:04.907859 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-a91bc308-5174-4040-b457-db3c33f9deb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a63507e4eb9f4dafbbb6aa5af971632c', 'request_id': 'req-a91bc308-5174-4040-b457-db3c33f9deb2'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:04.916454 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-a91bc308-5174-4040-b457-db3c33f9deb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a63507e4eb9f4dafbbb6aa5af971632c', 'request_id': 'req-a91bc308-5174-4040-b457-db3c33f9deb2'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:04.923890 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-a91bc308-5174-4040-b457-db3c33f9deb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a63507e4eb9f4dafbbb6aa5af971632c', 'request_id': 'req-a91bc308-5174-4040-b457-db3c33f9deb2'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:05.007499 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-a91bc308-5174-4040-b457-db3c33f9deb2 admin admin] The token cache is being invalidate because project a63507e4eb9f4dafbbb6aa5af971632c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:43:05.008250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-a91bc308-5174-4040-b457-db3c33f9deb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:05.009346 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-a91bc308-5174-4040-b457-db3c33f9deb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:43:05.011033 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1394/2802] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:43:04 2022] DELETE /identity/v3/projects/a63507e4eb9f4dafbbb6aa5af971632c => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:43:14.345102 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=78,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:15.000615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:18.984132 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e7ced-e6c3-403c-af7b-17fec66c654c None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:18.984704 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e7ced-e6c3-403c-af7b-17fec66c654c None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:18.984704 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e7ced-e6c3-403c-af7b-17fec66c654c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:19.007803 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-406e7ced-e6c3-403c-af7b-17fec66c654c None None] MFA Rules not processed for user `8492a5ac560f43e4a3342f067dda8fef`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:43:19.009048 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-406e7ced-e6c3-403c-af7b-17fec66c654c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:19.073997 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1409/2803] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:43:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:43:19.624953 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91bc308-5174-4040-b457-db3c33f9deb2 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:19.626661 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a91bc308-5174-4040-b457-db3c33f9deb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:19.713579 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91bc308-5174-4040-b457-db3c33f9deb2 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:19.717845 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315fe230-4a53-4753-b5c2-9dcecd12e5b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:19.718728 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fe230-4a53-4753-b5c2-9dcecd12e5b9 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:19.719042 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fe230-4a53-4753-b5c2-9dcecd12e5b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:19.719353 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fe230-4a53-4753-b5c2-9dcecd12e5b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:19.722153 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-315fe230-4a53-4753-b5c2-9dcecd12e5b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:19.734684 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315fe230-4a53-4753-b5c2-9dcecd12e5b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:19.736142 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315fe230-4a53-4753-b5c2-9dcecd12e5b9 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:19.746663 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1395/2804] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:43:29.066115 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=25 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:29.741989 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=26 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:43:40.079779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e7ced-e6c3-403c-af7b-17fec66c654c None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:43:40.109639 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e7ced-e6c3-403c-af7b-17fec66c654c None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:43:40.114287 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e69de-d2c1-4ece-9135-7b2aea0e63a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:43:40.114968 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e69de-d2c1-4ece-9135-7b2aea0e63a9 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:43:40.115246 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e69de-d2c1-4ece-9135-7b2aea0e63a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:43:40.115517 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e69de-d2c1-4ece-9135-7b2aea0e63a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:43:40.117438 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-ba8e69de-d2c1-4ece-9135-7b2aea0e63a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:43:40.210337 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e69de-d2c1-4ece-9135-7b2aea0e63a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:43:40.216941 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e69de-d2c1-4ece-9135-7b2aea0e63a9 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:43:40.224323 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1410/2805] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:43:40 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:43:50.217904 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=41 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:44:23.713469 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315fe230-4a53-4753-b5c2-9dcecd12e5b9 service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:44:23.717484 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-315fe230-4a53-4753-b5c2-9dcecd12e5b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:23.814962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315fe230-4a53-4753-b5c2-9dcecd12e5b9 service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:44:23.818977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c04ca-8034-4e6b-a972-14b849fa9ea6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:44:23.819637 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c04ca-8034-4e6b-a972-14b849fa9ea6 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:44:23.820167 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c04ca-8034-4e6b-a972-14b849fa9ea6 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:44:23.820499 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c04ca-8034-4e6b-a972-14b849fa9ea6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:44:23.822450 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-e24c04ca-8034-4e6b-a972-14b849fa9ea6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:44:23.909017 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c04ca-8034-4e6b-a972-14b849fa9ea6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:44:23.910521 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c04ca-8034-4e6b-a972-14b849fa9ea6 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:44:23.933161 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1396/2806] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:44:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:44:33.930018 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=65 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:46:01.408916 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e69de-d2c1-4ece-9135-7b2aea0e63a9 service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:46:01.445551 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e69de-d2c1-4ece-9135-7b2aea0e63a9 service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:46:01.452574 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77829861-c00b-44fb-a374-610de1cd6603 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:46:01.453598 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77829861-c00b-44fb-a374-610de1cd6603 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:01.454093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77829861-c00b-44fb-a374-610de1cd6603 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:01.454651 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77829861-c00b-44fb-a374-610de1cd6603 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:01.457801 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-77829861-c00b-44fb-a374-610de1cd6603 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:46:01.476396 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77829861-c00b-44fb-a374-610de1cd6603 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:46:01.480275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77829861-c00b-44fb-a374-610de1cd6603 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:46:01.497296 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1411/2807] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:46:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:46:09.965002 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c04ca-8034-4e6b-a972-14b849fa9ea6 service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:46:09.966742 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-e24c04ca-8034-4e6b-a972-14b849fa9ea6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:46:09.980813 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c04ca-8034-4e6b-a972-14b849fa9ea6 service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:46:09.984902 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca7fff-217f-429e-a4d5-ecd2ddf2cf21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:46:09.985535 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca7fff-217f-429e-a4d5-ecd2ddf2cf21 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:09.985816 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca7fff-217f-429e-a4d5-ecd2ddf2cf21 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:09.986089 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca7fff-217f-429e-a4d5-ecd2ddf2cf21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:09.988213 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-1bca7fff-217f-429e-a4d5-ecd2ddf2cf21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:46:10.078415 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca7fff-217f-429e-a4d5-ecd2ddf2cf21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:46:10.079879 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca7fff-217f-429e-a4d5-ecd2ddf2cf21 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:46:10.105265 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1397/2808] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:46:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:46:11.489365 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=10 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:46:12.913297 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77829861-c00b-44fb-a374-610de1cd6603 service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:46:12.914673 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-77829861-c00b-44fb-a374-610de1cd6603 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:46:12.936039 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77829861-c00b-44fb-a374-610de1cd6603 service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:46:12.945150 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a65534-9329-461d-bb76-8c5538558ddb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:46:12.945955 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a65534-9329-461d-bb76-8c5538558ddb service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:12.946338 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a65534-9329-461d-bb76-8c5538558ddb service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:12.947777 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a65534-9329-461d-bb76-8c5538558ddb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:12.951292 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-c7a65534-9329-461d-bb76-8c5538558ddb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:46:12.977125 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a65534-9329-461d-bb76-8c5538558ddb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:46:12.980810 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a65534-9329-461d-bb76-8c5538558ddb service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:46:13.006679 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1412/2809] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:46:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:46:14.800722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1b27b-2cb3-4cfe-be9a-676f0c193680 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:14.807776 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1b27b-2cb3-4cfe-be9a-676f0c193680 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:14.807776 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1b27b-2cb3-4cfe-be9a-676f0c193680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:14.825099 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-7dc1b27b-2cb3-4cfe-be9a-676f0c193680 None None] MFA Rules not processed for user `8492a5ac560f43e4a3342f067dda8fef`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:46:14.826433 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-7dc1b27b-2cb3-4cfe-be9a-676f0c193680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:46:14.833993 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1398/2810] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:46:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:46:15.260031 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a65534-9329-461d-bb76-8c5538558ddb service nova] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:46:15.274316 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a65534-9329-461d-bb76-8c5538558ddb service nova] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:46:15.280935 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73dd489-89c8-46e0-916d-b0fe8df92f76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:46:15.281915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73dd489-89c8-46e0-916d-b0fe8df92f76 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:15.282351 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73dd489-89c8-46e0-916d-b0fe8df92f76 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:15.282828 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73dd489-89c8-46e0-916d-b0fe8df92f76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:15.285495 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a73dd489-89c8-46e0-916d-b0fe8df92f76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:46:15.301699 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73dd489-89c8-46e0-916d-b0fe8df92f76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:46:15.303580 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73dd489-89c8-46e0-916d-b0fe8df92f76 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:46:15.314133 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1413/2811] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:46:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:46:16.205384 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1b27b-2cb3-4cfe-be9a-676f0c193680 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:46:16.207317 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-7dc1b27b-2cb3-4cfe-be9a-676f0c193680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:46:16.312270 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1b27b-2cb3-4cfe-be9a-676f0c193680 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:46:16.318385 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899200ae-897b-4fd1-a4fb-7a4bcd415627 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3db46a4b4e74cc0bb2a7bd6f0e99105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:46:16.319062 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899200ae-897b-4fd1-a4fb-7a4bcd415627 service placement] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:16.319342 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899200ae-897b-4fd1-a4fb-7a4bcd415627 service placement] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:16.319638 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899200ae-897b-4fd1-a4fb-7a4bcd415627 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:16.321649 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-899200ae-897b-4fd1-a4fb-7a4bcd415627 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:46:16.336284 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899200ae-897b-4fd1-a4fb-7a4bcd415627 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:46:16.339481 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899200ae-897b-4fd1-a4fb-7a4bcd415627 service placement] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:46:16.358856 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1399/2812] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:46:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:46:25.309234 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=12 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:46:26.349804 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=76 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:46:33.316283 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73dd489-89c8-46e0-916d-b0fe8df92f76 service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:46:33.340242 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73dd489-89c8-46e0-916d-b0fe8df92f76 service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:46:33.350323 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26593e1-c3eb-4034-ab86-12a80f04ff92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:46:33.351797 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26593e1-c3eb-4034-ab86-12a80f04ff92 service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:46:33.352785 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26593e1-c3eb-4034-ab86-12a80f04ff92 service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:46:33.353274 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26593e1-c3eb-4034-ab86-12a80f04ff92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:46:33.356520 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-f26593e1-c3eb-4034-ab86-12a80f04ff92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:46:33.380571 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26593e1-c3eb-4034-ab86-12a80f04ff92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:46:33.383897 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26593e1-c3eb-4034-ab86-12a80f04ff92 service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:46:33.416013 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1414/2813] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:46:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:46:43.396075 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=6 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:47:02.691444 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899200ae-897b-4fd1-a4fb-7a4bcd415627 service placement] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:47:02.704334 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899200ae-897b-4fd1-a4fb-7a4bcd415627 service placement] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:47:02.710693 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d12c50a-49ef-456e-8713-4ec977485573 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3db46a4b4e74cc0bb2a7bd6f0e99105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:47:02.711392 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12c50a-49ef-456e-8713-4ec977485573 service placement] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:47:02.711903 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12c50a-49ef-456e-8713-4ec977485573 service placement] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:47:02.712302 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12c50a-49ef-456e-8713-4ec977485573 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:47:02.714482 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-4d12c50a-49ef-456e-8713-4ec977485573 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:47:02.727645 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d12c50a-49ef-456e-8713-4ec977485573 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:47:02.729293 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d12c50a-49ef-456e-8713-4ec977485573 service placement] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:47:02.740803 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1400/2814] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:47:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:47:12.735870 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=6 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:48:38.638978 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26593e1-c3eb-4034-ab86-12a80f04ff92 service nova] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:48:38.649484 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26593e1-c3eb-4034-ab86-12a80f04ff92 service nova] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:48:38.653160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f0fae-15ca-4234-947e-48bd4cfd19dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:48:38.653772 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f0fae-15ca-4234-947e-48bd4cfd19dc service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:48:38.654047 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f0fae-15ca-4234-947e-48bd4cfd19dc service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:48:38.654308 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f0fae-15ca-4234-947e-48bd4cfd19dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:48:38.656180 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-043f0fae-15ca-4234-947e-48bd4cfd19dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:48:38.667744 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f0fae-15ca-4234-947e-48bd4cfd19dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:48:38.669176 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f0fae-15ca-4234-947e-48bd4cfd19dc service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:48:38.678692 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1415/2815] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:48:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:48:48.674666 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=6 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:48:52.691060 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999974e-f458-401e-8614-5773360621c3 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:48:52.691970 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999974e-f458-401e-8614-5773360621c3 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:48:52.691970 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999974e-f458-401e-8614-5773360621c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:48:52.727569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-d999974e-f458-401e-8614-5773360621c3 None None] MFA Rules not processed for user `8492a5ac560f43e4a3342f067dda8fef`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:48:52.728996 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-d999974e-f458-401e-8614-5773360621c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:48:52.737560 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1401/2816] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:48:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:48:54.013270 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f0fae-15ca-4234-947e-48bd4cfd19dc service cinder] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:48:54.026783 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f0fae-15ca-4234-947e-48bd4cfd19dc service cinder] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:48:54.034595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520d462-80e8-42fd-839e-af615192f550 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:48:54.035948 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520d462-80e8-42fd-839e-af615192f550 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:48:54.036152 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520d462-80e8-42fd-839e-af615192f550 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:48:54.036636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520d462-80e8-42fd-839e-af615192f550 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:48:54.039794 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-b520d462-80e8-42fd-839e-af615192f550 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:48:54.069706 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520d462-80e8-42fd-839e-af615192f550 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:48:54.072623 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520d462-80e8-42fd-839e-af615192f550 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:48:54.102339 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1416/2817] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:48:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:48:57.118628 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999974e-f458-401e-8614-5773360621c3 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:48:57.131702 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999974e-f458-401e-8614-5773360621c3 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:48:57.137363 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51c69c-3f1a-424c-b66b-ac9a4cec4aad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:48:57.138090 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51c69c-3f1a-424c-b66b-ac9a4cec4aad service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:48:57.138531 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51c69c-3f1a-424c-b66b-ac9a4cec4aad service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:48:57.138905 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51c69c-3f1a-424c-b66b-ac9a4cec4aad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:48:57.141277 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-0a51c69c-3f1a-424c-b66b-ac9a4cec4aad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:48:57.254176 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a51c69c-3f1a-424c-b66b-ac9a4cec4aad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:48:57.257103 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a51c69c-3f1a-424c-b66b-ac9a4cec4aad service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:48:57.273719 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1402/2818] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:48:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:49:01.847697 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520d462-80e8-42fd-839e-af615192f550 service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:01.852484 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-b520d462-80e8-42fd-839e-af615192f550 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:01.883183 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520d462-80e8-42fd-839e-af615192f550 service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:01.893492 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c39f3-666e-4d67-afc7-d33cc9b92f64 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3db46a4b4e74cc0bb2a7bd6f0e99105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:01.894741 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c39f3-666e-4d67-afc7-d33cc9b92f64 service placement] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:01.895160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c39f3-666e-4d67-afc7-d33cc9b92f64 service placement] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:01.895600 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c39f3-666e-4d67-afc7-d33cc9b92f64 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:01.898415 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-315c39f3-666e-4d67-afc7-d33cc9b92f64 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:01.918157 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c39f3-666e-4d67-afc7-d33cc9b92f64 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:01.919926 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c39f3-666e-4d67-afc7-d33cc9b92f64 service placement] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:01.932653 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1417/2819] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:49:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:49:01.999728 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51c69c-3f1a-424c-b66b-ac9a4cec4aad service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:02.003433 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-0a51c69c-3f1a-424c-b66b-ac9a4cec4aad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:02.026410 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51c69c-3f1a-424c-b66b-ac9a4cec4aad service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:02.034920 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97677f78-9172-42e0-97df-690e279c800c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:02.035973 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97677f78-9172-42e0-97df-690e279c800c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:02.036442 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97677f78-9172-42e0-97df-690e279c800c admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:02.036923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97677f78-9172-42e0-97df-690e279c800c admin admin] PATH_INFO: `/v3/users/d5d0010f584249b4a22fd31634c9fd8b` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:02.038979 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97677f78-9172-42e0-97df-690e279c800c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5d0010f584249b4a22fd31634c9fd8b)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:02.042592 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97677f78-9172-42e0-97df-690e279c800c admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:02.355612 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-97677f78-9172-42e0-97df-690e279c800c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5d0010f584249b4a22fd31634c9fd8b'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:02.431161 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-97677f78-9172-42e0-97df-690e279c800c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5d0010f584249b4a22fd31634c9fd8b'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:02.431796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-97677f78-9172-42e0-97df-690e279c800c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5d0010f584249b4a22fd31634c9fd8b'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:02.432431 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-97677f78-9172-42e0-97df-690e279c800c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5d0010f584249b4a22fd31634c9fd8b'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:02.447911 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-97677f78-9172-42e0-97df-690e279c800c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5d0010f584249b4a22fd31634c9fd8b'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:02.460634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1403/2820] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:49:01 2022] DELETE /identity/v3/users/d5d0010f584249b4a22fd31634c9fd8b => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:49:02.743129 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c39f3-666e-4d67-afc7-d33cc9b92f64 service placement] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:02.755890 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c39f3-666e-4d67-afc7-d33cc9b92f64 service placement] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:02.834918 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b44fc4-987b-4675-89a8-5d803fc730ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:02.835731 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b44fc4-987b-4675-89a8-5d803fc730ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:02.836253 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b44fc4-987b-4675-89a8-5d803fc730ba admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:02.836672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b44fc4-987b-4675-89a8-5d803fc730ba admin admin] PATH_INFO: `/v3/projects/1d62244979394d21a86fdbbe6a56ad36` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:02.837948 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b44fc4-987b-4675-89a8-5d803fc730ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d62244979394d21a86fdbbe6a56ad36)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:02.840039 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b44fc4-987b-4675-89a8-5d803fc730ba admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:02.971145 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e2b44fc4-987b-4675-89a8-5d803fc730ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d62244979394d21a86fdbbe6a56ad36', 'request_id': 'req-e2b44fc4-987b-4675-89a8-5d803fc730ba'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:03.053900 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e2b44fc4-987b-4675-89a8-5d803fc730ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d62244979394d21a86fdbbe6a56ad36', 'request_id': 'req-e2b44fc4-987b-4675-89a8-5d803fc730ba'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:03.061012 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e2b44fc4-987b-4675-89a8-5d803fc730ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d62244979394d21a86fdbbe6a56ad36', 'request_id': 'req-e2b44fc4-987b-4675-89a8-5d803fc730ba'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:03.074458 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e2b44fc4-987b-4675-89a8-5d803fc730ba admin admin] The token cache is being invalidate because project 1d62244979394d21a86fdbbe6a56ad36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:49:03.074751 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e2b44fc4-987b-4675-89a8-5d803fc730ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:03.075465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-e2b44fc4-987b-4675-89a8-5d803fc730ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:03.076982 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1418/2821] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:49:02 2022] DELETE /identity/v3/projects/1d62244979394d21a86fdbbe6a56ad36 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:49:03.085499 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5f43b-fcbc-41fe-a1d6-bf89636700d4 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:03.086485 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5f43b-fcbc-41fe-a1d6-bf89636700d4 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:03.086665 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5f43b-fcbc-41fe-a1d6-bf89636700d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:03.110531 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-33e5f43b-fcbc-41fe-a1d6-bf89636700d4 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:03.111960 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-33e5f43b-fcbc-41fe-a1d6-bf89636700d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:03.178364 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:03.187871 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1404/2822] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:49:03.196564 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ad3a6-f65d-4898-9524-413be34f5f7f None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:03.196857 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ad3a6-f65d-4898-9524-413be34f5f7f None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:03.197314 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ad3a6-f65d-4898-9524-413be34f5f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:03.230142 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-680ad3a6-f65d-4898-9524-413be34f5f7f None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:03.232313 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-680ad3a6-f65d-4898-9524-413be34f5f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:03.242168 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1419/2823] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:49:03.252399 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5f43b-fcbc-41fe-a1d6-bf89636700d4 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:03.268237 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5f43b-fcbc-41fe-a1d6-bf89636700d4 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:03.276131 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33105f08-b675-4b1a-a329-5d9968ab5a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:03.276808 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33105f08-b675-4b1a-a329-5d9968ab5a4a admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:03.277083 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33105f08-b675-4b1a-a329-5d9968ab5a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:03.277351 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33105f08-b675-4b1a-a329-5d9968ab5a4a admin admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:03.278003 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33105f08-b675-4b1a-a329-5d9968ab5a4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:49:03.278305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33105f08-b675-4b1a-a329-5d9968ab5a4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:03.280094 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33105f08-b675-4b1a-a329-5d9968ab5a4a admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:03.293809 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1405/2824] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:49:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:49:03.298927 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ad3a6-f65d-4898-9524-413be34f5f7f None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:03.308913 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ad3a6-f65d-4898-9524-413be34f5f7f None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:03.314870 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77faff-6fdf-405b-93e0-76bbb13fefa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:03.315502 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77faff-6fdf-405b-93e0-76bbb13fefa8 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:03.315783 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77faff-6fdf-405b-93e0-76bbb13fefa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:03.316052 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77faff-6fdf-405b-93e0-76bbb13fefa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:03.316721 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec77faff-6fdf-405b-93e0-76bbb13fefa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:03.318699 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec77faff-6fdf-405b-93e0-76bbb13fefa8 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:03.491312 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1420/2825] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:49:03 2022] POST /identity/v3/projects => generated 387 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:49:03.495721 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33105f08-b675-4b1a-a329-5d9968ab5a4a admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:03.507904 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33105f08-b675-4b1a-a329-5d9968ab5a4a admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:03.573869 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ea1c2-2fe5-4425-946d-f5b5fa973ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:03.574659 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ea1c2-2fe5-4425-946d-f5b5fa973ba7 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:03.574947 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ea1c2-2fe5-4425-946d-f5b5fa973ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:03.575222 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ea1c2-2fe5-4425-946d-f5b5fa973ba7 admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:03.575921 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5ea1c2-2fe5-4425-946d-f5b5fa973ba7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:03.577876 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5ea1c2-2fe5-4425-946d-f5b5fa973ba7 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:03.723828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1406/2826] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:49:03 2022] POST /identity/v3/users => generated 294 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:49:03.728057 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77faff-6fdf-405b-93e0-76bbb13fefa8 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:03.739676 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77faff-6fdf-405b-93e0-76bbb13fefa8 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:03.745000 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683fff80-acb6-4a1d-8d61-7df5161a1bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:03.745544 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fff80-acb6-4a1d-8d61-7df5161a1bfa admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:03.745854 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fff80-acb6-4a1d-8d61-7df5161a1bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:03.746125 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fff80-acb6-4a1d-8d61-7df5161a1bfa admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:03.746813 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683fff80-acb6-4a1d-8d61-7df5161a1bfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:03.748505 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683fff80-acb6-4a1d-8d61-7df5161a1bfa admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:03.763140 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1421/2827] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:49:03.768061 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ea1c2-2fe5-4425-946d-f5b5fa973ba7 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:03.779209 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ea1c2-2fe5-4425-946d-f5b5fa973ba7 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:03.784622 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be5ff3-7acc-4a74-9b07-4c70c5a5a609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:03.785274 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be5ff3-7acc-4a74-9b07-4c70c5a5a609 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:03.785555 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be5ff3-7acc-4a74-9b07-4c70c5a5a609 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:03.785821 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be5ff3-7acc-4a74-9b07-4c70c5a5a609 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:03.786506 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be5ff3-7acc-4a74-9b07-4c70c5a5a609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:03.788247 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be5ff3-7acc-4a74-9b07-4c70c5a5a609 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:03.800958 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1407/2828] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:49:03.808371 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683fff80-acb6-4a1d-8d61-7df5161a1bfa admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:03.822710 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683fff80-acb6-4a1d-8d61-7df5161a1bfa admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:03.827595 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b87ab-aa2a-4fa2-b296-272cd94736be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:03.828261 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b87ab-aa2a-4fa2-b296-272cd94736be admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:03.828537 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b87ab-aa2a-4fa2-b296-272cd94736be admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:03.828816 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b87ab-aa2a-4fa2-b296-272cd94736be admin admin] PATH_INFO: `/v3/projects/568ddda4680046488b9acf01afc1c454/users/a70411a7a1944a699f19431d1fcde85b/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:03.846953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b87ab-aa2a-4fa2-b296-272cd94736be admin admin] RBAC: Authorizing `identity:create_grant(project_id=568ddda4680046488b9acf01afc1c454, user_id=a70411a7a1944a699f19431d1fcde85b, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:03.849660 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b87ab-aa2a-4fa2-b296-272cd94736be admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:03.981329 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:03.986945 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1422/2829] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:49:03 2022] PUT /identity/v3/projects/568ddda4680046488b9acf01afc1c454/users/a70411a7a1944a699f19431d1fcde85b/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:49:03.998774 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4aa5a-f07c-433d-928e-bab23cc9f548 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:03.998977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4aa5a-f07c-433d-928e-bab23cc9f548 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:03.999237 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4aa5a-f07c-433d-928e-bab23cc9f548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:04.034862 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-98b4aa5a-f07c-433d-928e-bab23cc9f548 None None] MFA Rules not processed for user `a70411a7a1944a699f19431d1fcde85b`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:04.036221 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-98b4aa5a-f07c-433d-928e-bab23cc9f548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:04.109777 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1408/2830] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:49:04.115301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b87ab-aa2a-4fa2-b296-272cd94736be admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:04.196902 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b87ab-aa2a-4fa2-b296-272cd94736be admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:04.202767 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b1eed-887a-415a-ae6a-9ed7efe1c875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:04.203436 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b1eed-887a-415a-ae6a-9ed7efe1c875 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:04.203760 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b1eed-887a-415a-ae6a-9ed7efe1c875 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:04.204688 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b1eed-887a-415a-ae6a-9ed7efe1c875 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:04.205523 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1b1eed-887a-415a-ae6a-9ed7efe1c875 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:04.208614 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1b1eed-887a-415a-ae6a-9ed7efe1c875 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:04.348903 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1423/2831] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:49:04 2022] POST /identity/v3/projects => generated 385 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:49:04.354160 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4aa5a-f07c-433d-928e-bab23cc9f548 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:04.365176 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4aa5a-f07c-433d-928e-bab23cc9f548 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:04.424807 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:04.433796 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077f3c0-c0c3-412c-86d4-424f7e5d7367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:04.434891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077f3c0-c0c3-412c-86d4-424f7e5d7367 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:04.435271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077f3c0-c0c3-412c-86d4-424f7e5d7367 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:04.435733 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077f3c0-c0c3-412c-86d4-424f7e5d7367 admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:04.436457 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077f3c0-c0c3-412c-86d4-424f7e5d7367 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:04.438684 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077f3c0-c0c3-412c-86d4-424f7e5d7367 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:04.568615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1409/2832] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:49:04 2022] POST /identity/v3/users => generated 299 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:49:04.570616 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b1eed-887a-415a-ae6a-9ed7efe1c875 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:04.582079 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b1eed-887a-415a-ae6a-9ed7efe1c875 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:04.587215 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d198f7-f885-4635-9f01-4873464aae92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:04.588017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d198f7-f885-4635-9f01-4873464aae92 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:04.588187 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d198f7-f885-4635-9f01-4873464aae92 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:04.588377 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d198f7-f885-4635-9f01-4873464aae92 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:04.589040 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d198f7-f885-4635-9f01-4873464aae92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:04.590731 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d198f7-f885-4635-9f01-4873464aae92 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:04.604775 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1424/2833] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:49:04.610952 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077f3c0-c0c3-412c-86d4-424f7e5d7367 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:04.627848 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077f3c0-c0c3-412c-86d4-424f7e5d7367 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:04.633773 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207c279-3a61-49d8-a232-2f23a6d8b4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:04.634514 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207c279-3a61-49d8-a232-2f23a6d8b4f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:04.634795 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207c279-3a61-49d8-a232-2f23a6d8b4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:04.635074 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207c279-3a61-49d8-a232-2f23a6d8b4f6 admin admin] PATH_INFO: `/v3/projects/3bbea819757041caa5a22587f743cc2b/users/2705c0f4b03f44c9b150be6a1c4b4b59/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:04.653465 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207c279-3a61-49d8-a232-2f23a6d8b4f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bbea819757041caa5a22587f743cc2b, user_id=2705c0f4b03f44c9b150be6a1c4b4b59, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:04.656262 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207c279-3a61-49d8-a232-2f23a6d8b4f6 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:04.748581 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1410/2834] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:49:04 2022] PUT /identity/v3/projects/3bbea819757041caa5a22587f743cc2b/users/2705c0f4b03f44c9b150be6a1c4b4b59/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:49:04.753771 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d198f7-f885-4635-9f01-4873464aae92 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:04.772723 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d198f7-f885-4635-9f01-4873464aae92 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:04.887641 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86a794-ad75-4ecb-93fa-db27275fd110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:04.888714 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86a794-ad75-4ecb-93fa-db27275fd110 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:04.889218 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86a794-ad75-4ecb-93fa-db27275fd110 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:04.889714 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86a794-ad75-4ecb-93fa-db27275fd110 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:04.891002 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86a794-ad75-4ecb-93fa-db27275fd110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:04.897820 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86a794-ad75-4ecb-93fa-db27275fd110 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:04.923340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1425/2835] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:49:04.925068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207c279-3a61-49d8-a232-2f23a6d8b4f6 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:04.938843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207c279-3a61-49d8-a232-2f23a6d8b4f6 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:04.945766 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d085ec4-970e-4ce3-aa56-de7581509339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:04.947774 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d085ec4-970e-4ce3-aa56-de7581509339 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:04.947774 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d085ec4-970e-4ce3-aa56-de7581509339 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:04.947774 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d085ec4-970e-4ce3-aa56-de7581509339 admin admin] PATH_INFO: `/v3/projects/3bbea819757041caa5a22587f743cc2b/users/2705c0f4b03f44c9b150be6a1c4b4b59/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:04.952292 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d085ec4-970e-4ce3-aa56-de7581509339 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bbea819757041caa5a22587f743cc2b, user_id=2705c0f4b03f44c9b150be6a1c4b4b59, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:04.953886 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d085ec4-970e-4ce3-aa56-de7581509339 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:04.969099 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1411/2836] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:49:04 2022] PUT /identity/v3/projects/3bbea819757041caa5a22587f743cc2b/users/2705c0f4b03f44c9b150be6a1c4b4b59/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:49:04.981072 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7f61a-de78-4109-9dd0-daa2b7b6aaa4 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:04.981436 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7f61a-de78-4109-9dd0-daa2b7b6aaa4 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:04.982126 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7f61a-de78-4109-9dd0-daa2b7b6aaa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:05.013828 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-e1a7f61a-de78-4109-9dd0-daa2b7b6aaa4 None None] MFA Rules not processed for user `2705c0f4b03f44c9b150be6a1c4b4b59`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:05.015202 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-e1a7f61a-de78-4109-9dd0-daa2b7b6aaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:05.084756 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:05.095720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1426/2837] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:04 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:49:05.108133 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d085ec4-970e-4ce3-aa56-de7581509339 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:05.111638 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-4d085ec4-970e-4ce3-aa56-de7581509339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:05.180281 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d085ec4-970e-4ce3-aa56-de7581509339 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:05.184445 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558beb11-2024-4a22-8465-57423d992cea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:05.185344 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558beb11-2024-4a22-8465-57423d992cea service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:05.185768 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558beb11-2024-4a22-8465-57423d992cea service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:05.186079 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558beb11-2024-4a22-8465-57423d992cea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:05.197364 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558beb11-2024-4a22-8465-57423d992cea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:05.198855 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558beb11-2024-4a22-8465-57423d992cea service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:05.217653 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1412/2838] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:49:05 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:49:05.256740 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7f61a-de78-4109-9dd0-daa2b7b6aaa4 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:05.357870 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7f61a-de78-4109-9dd0-daa2b7b6aaa4 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:05.365354 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c23270-0342-4e08-b0ca-fc41268d2d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:05.366386 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c23270-0342-4e08-b0ca-fc41268d2d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:05.366872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c23270-0342-4e08-b0ca-fc41268d2d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:05.367305 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c23270-0342-4e08-b0ca-fc41268d2d9d admin admin] PATH_INFO: `/v3/users/a70411a7a1944a699f19431d1fcde85b` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:05.369319 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c23270-0342-4e08-b0ca-fc41268d2d9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a70411a7a1944a699f19431d1fcde85b)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:05.372757 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c23270-0342-4e08-b0ca-fc41268d2d9d admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:05.572004 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-59c23270-0342-4e08-b0ca-fc41268d2d9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a70411a7a1944a699f19431d1fcde85b'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:05.587496 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-59c23270-0342-4e08-b0ca-fc41268d2d9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a70411a7a1944a699f19431d1fcde85b'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:05.596155 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-59c23270-0342-4e08-b0ca-fc41268d2d9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a70411a7a1944a699f19431d1fcde85b'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:05.680936 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-59c23270-0342-4e08-b0ca-fc41268d2d9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a70411a7a1944a699f19431d1fcde85b'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:05.681566 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-59c23270-0342-4e08-b0ca-fc41268d2d9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a70411a7a1944a699f19431d1fcde85b'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:05.688987 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1427/2839] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:49:05 2022] DELETE /identity/v3/users/a70411a7a1944a699f19431d1fcde85b => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:49:05.694963 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df10706-c884-454e-bee3-c4c170028408 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:05.695464 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df10706-c884-454e-bee3-c4c170028408 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:05.695984 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df10706-c884-454e-bee3-c4c170028408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:05.727130 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-8df10706-c884-454e-bee3-c4c170028408 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:05.729113 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-8df10706-c884-454e-bee3-c4c170028408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:05.839197 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1413/2840] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:49:05.850299 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c23270-0342-4e08-b0ca-fc41268d2d9d admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:05.851733 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-59c23270-0342-4e08-b0ca-fc41268d2d9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:05.921254 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c23270-0342-4e08-b0ca-fc41268d2d9d admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:05.925713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be32d709-961b-4748-88e1-d5e2ba65f6d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:05.926334 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be32d709-961b-4748-88e1-d5e2ba65f6d6 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:05.926641 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be32d709-961b-4748-88e1-d5e2ba65f6d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:05.926909 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be32d709-961b-4748-88e1-d5e2ba65f6d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:05.937389 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be32d709-961b-4748-88e1-d5e2ba65f6d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:05.939560 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be32d709-961b-4748-88e1-d5e2ba65f6d6 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:05.954361 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1428/2841] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:49:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:49:06.455240 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df10706-c884-454e-bee3-c4c170028408 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:06.457576 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-8df10706-c884-454e-bee3-c4c170028408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:06.476612 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df10706-c884-454e-bee3-c4c170028408 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:06.483832 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa999c-0616-4777-b713-eb502f7abd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:06.484696 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa999c-0616-4777-b713-eb502f7abd7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:06.484919 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa999c-0616-4777-b713-eb502f7abd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:06.485421 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa999c-0616-4777-b713-eb502f7abd7a admin admin] PATH_INFO: `/v3/projects/568ddda4680046488b9acf01afc1c454` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:06.486566 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa999c-0616-4777-b713-eb502f7abd7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=568ddda4680046488b9acf01afc1c454)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:06.488498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa999c-0616-4777-b713-eb502f7abd7a admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:06.612269 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:06.617723 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2faa999c-0616-4777-b713-eb502f7abd7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '568ddda4680046488b9acf01afc1c454', 'request_id': 'req-2faa999c-0616-4777-b713-eb502f7abd7a'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:06.631193 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2faa999c-0616-4777-b713-eb502f7abd7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '568ddda4680046488b9acf01afc1c454', 'request_id': 'req-2faa999c-0616-4777-b713-eb502f7abd7a'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:06.641543 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2faa999c-0616-4777-b713-eb502f7abd7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '568ddda4680046488b9acf01afc1c454', 'request_id': 'req-2faa999c-0616-4777-b713-eb502f7abd7a'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:06.758666 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2faa999c-0616-4777-b713-eb502f7abd7a admin admin] The token cache is being invalidate because project 568ddda4680046488b9acf01afc1c454 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:49:06.758962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2faa999c-0616-4777-b713-eb502f7abd7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:06.759664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2faa999c-0616-4777-b713-eb502f7abd7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:06.761233 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1414/2842] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:49:06 2022] DELETE /identity/v3/projects/568ddda4680046488b9acf01afc1c454 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:49:06.765554 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be32d709-961b-4748-88e1-d5e2ba65f6d6 service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:06.776320 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be32d709-961b-4748-88e1-d5e2ba65f6d6 service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:06.790946 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:06.842459 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f6792-710c-4b59-9ab5-6a5e7b2489f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:06.843091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f6792-710c-4b59-9ab5-6a5e7b2489f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:06.843361 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f6792-710c-4b59-9ab5-6a5e7b2489f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:06.843627 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f6792-710c-4b59-9ab5-6a5e7b2489f4 admin admin] PATH_INFO: `/v3/users/2705c0f4b03f44c9b150be6a1c4b4b59` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:06.844878 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f6792-710c-4b59-9ab5-6a5e7b2489f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2705c0f4b03f44c9b150be6a1c4b4b59)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:06.846976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f6792-710c-4b59-9ab5-6a5e7b2489f4 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:07.253849 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a86f6792-710c-4b59-9ab5-6a5e7b2489f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2705c0f4b03f44c9b150be6a1c4b4b59'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:07.275649 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a86f6792-710c-4b59-9ab5-6a5e7b2489f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2705c0f4b03f44c9b150be6a1c4b4b59'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:07.283460 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a86f6792-710c-4b59-9ab5-6a5e7b2489f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2705c0f4b03f44c9b150be6a1c4b4b59'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:07.400930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a86f6792-710c-4b59-9ab5-6a5e7b2489f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2705c0f4b03f44c9b150be6a1c4b4b59'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:07.401651 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-a86f6792-710c-4b59-9ab5-6a5e7b2489f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2705c0f4b03f44c9b150be6a1c4b4b59'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:07.408794 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1429/2843] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:49:06 2022] DELETE /identity/v3/users/2705c0f4b03f44c9b150be6a1c4b4b59 => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:49:07.976289 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa999c-0616-4777-b713-eb502f7abd7a admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:07.978525 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-2faa999c-0616-4777-b713-eb502f7abd7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:07.996736 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa999c-0616-4777-b713-eb502f7abd7a admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:08.086210 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a6fa0-0534-4756-ad46-d56f516532e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:08.086896 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a6fa0-0534-4756-ad46-d56f516532e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:08.087178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a6fa0-0534-4756-ad46-d56f516532e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:08.087452 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a6fa0-0534-4756-ad46-d56f516532e1 admin admin] PATH_INFO: `/v3/projects/3bbea819757041caa5a22587f743cc2b` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:08.088668 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7a6fa0-0534-4756-ad46-d56f516532e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bbea819757041caa5a22587f743cc2b)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:08.090645 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7a6fa0-0534-4756-ad46-d56f516532e1 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:08.214707 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-1e7a6fa0-0534-4756-ad46-d56f516532e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bbea819757041caa5a22587f743cc2b', 'request_id': 'req-1e7a6fa0-0534-4756-ad46-d56f516532e1'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:08.222077 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-1e7a6fa0-0534-4756-ad46-d56f516532e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bbea819757041caa5a22587f743cc2b', 'request_id': 'req-1e7a6fa0-0534-4756-ad46-d56f516532e1'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:08.228528 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-1e7a6fa0-0534-4756-ad46-d56f516532e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bbea819757041caa5a22587f743cc2b', 'request_id': 'req-1e7a6fa0-0534-4756-ad46-d56f516532e1'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:08.311646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-1e7a6fa0-0534-4756-ad46-d56f516532e1 admin admin] The token cache is being invalidate because project 3bbea819757041caa5a22587f743cc2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:49:08.311945 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-1e7a6fa0-0534-4756-ad46-d56f516532e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:08.312767 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-1e7a6fa0-0534-4756-ad46-d56f516532e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:08.315914 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1415/2844] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:49:07 2022] DELETE /identity/v3/projects/3bbea819757041caa5a22587f743cc2b => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:49:08.321699 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b08034-4d7d-45f9-8b14-8da25f142d8a None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:08.322205 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b08034-4d7d-45f9-8b14-8da25f142d8a None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:08.322670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b08034-4d7d-45f9-8b14-8da25f142d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:08.349727 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-e7b08034-4d7d-45f9-8b14-8da25f142d8a None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:08.351221 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-e7b08034-4d7d-45f9-8b14-8da25f142d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:08.417414 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1430/2845] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:49:08.426010 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e839525-57d8-403b-a4a0-fc6c138f4c24 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:08.426565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e839525-57d8-403b-a4a0-fc6c138f4c24 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:08.426850 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e839525-57d8-403b-a4a0-fc6c138f4c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:08.448832 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-4e839525-57d8-403b-a4a0-fc6c138f4c24 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:08.452293 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-4e839525-57d8-403b-a4a0-fc6c138f4c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:08.459515 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1416/2846] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:49:08.464086 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b08034-4d7d-45f9-8b14-8da25f142d8a None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:08.476047 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b08034-4d7d-45f9-8b14-8da25f142d8a None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:08.480740 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c6690-b66c-4d25-be5d-f35af596625c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:08.481380 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c6690-b66c-4d25-be5d-f35af596625c admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:08.481666 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c6690-b66c-4d25-be5d-f35af596625c admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:08.481935 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c6690-b66c-4d25-be5d-f35af596625c admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:08.482636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c6690-b66c-4d25-be5d-f35af596625c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:49:08.482942 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c6690-b66c-4d25-be5d-f35af596625c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:08.484578 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c6690-b66c-4d25-be5d-f35af596625c admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:08.498706 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1431/2847] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:49:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:49:08.506372 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e839525-57d8-403b-a4a0-fc6c138f4c24 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:08.523486 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e839525-57d8-403b-a4a0-fc6c138f4c24 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:08.530510 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda85ce5-4e37-4359-b007-53872ccd6e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:08.531550 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda85ce5-4e37-4359-b007-53872ccd6e31 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:08.531958 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda85ce5-4e37-4359-b007-53872ccd6e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:08.532352 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda85ce5-4e37-4359-b007-53872ccd6e31 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:08.533143 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda85ce5-4e37-4359-b007-53872ccd6e31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:08.535992 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda85ce5-4e37-4359-b007-53872ccd6e31 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:08.645513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1417/2848] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:49:08 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:49:08.651507 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c6690-b66c-4d25-be5d-f35af596625c admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:08.665050 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c6690-b66c-4d25-be5d-f35af596625c admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:08.689312 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:08.732234 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc57bdba-455b-4c8a-a68e-8ace805333fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:08.732825 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57bdba-455b-4c8a-a68e-8ace805333fc admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:08.733101 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57bdba-455b-4c8a-a68e-8ace805333fc admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:08.733369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57bdba-455b-4c8a-a68e-8ace805333fc admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:08.734017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc57bdba-455b-4c8a-a68e-8ace805333fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:08.737077 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc57bdba-455b-4c8a-a68e-8ace805333fc admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:08.843683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1432/2849] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:49:08 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:49:08.849228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda85ce5-4e37-4359-b007-53872ccd6e31 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:08.868490 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda85ce5-4e37-4359-b007-53872ccd6e31 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:08.877305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a7bc0-1a47-450c-8c90-60d62caea0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:08.878413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a7bc0-1a47-450c-8c90-60d62caea0ba admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:08.878941 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a7bc0-1a47-450c-8c90-60d62caea0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:08.879418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a7bc0-1a47-450c-8c90-60d62caea0ba admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:08.880541 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5a7bc0-1a47-450c-8c90-60d62caea0ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:08.883759 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5a7bc0-1a47-450c-8c90-60d62caea0ba admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:08.901017 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1418/2850] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:49:08.906819 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc57bdba-455b-4c8a-a68e-8ace805333fc admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:08.917976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc57bdba-455b-4c8a-a68e-8ace805333fc admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:08.923565 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118c41c-9566-45fd-b72b-766ec5c1cd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:08.924214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118c41c-9566-45fd-b72b-766ec5c1cd76 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:08.924492 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118c41c-9566-45fd-b72b-766ec5c1cd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:08.924761 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118c41c-9566-45fd-b72b-766ec5c1cd76 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:08.925414 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5118c41c-9566-45fd-b72b-766ec5c1cd76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:08.927103 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5118c41c-9566-45fd-b72b-766ec5c1cd76 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:08.940007 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1433/2851] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:49:08.944343 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a7bc0-1a47-450c-8c90-60d62caea0ba admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:08.955653 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a7bc0-1a47-450c-8c90-60d62caea0ba admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:08.960684 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69fc206-4ab5-43eb-bab5-ba7efc341aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:08.961346 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69fc206-4ab5-43eb-bab5-ba7efc341aae admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:08.961621 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69fc206-4ab5-43eb-bab5-ba7efc341aae admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:08.961888 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69fc206-4ab5-43eb-bab5-ba7efc341aae admin admin] PATH_INFO: `/v3/projects/8744e2ccdebc4c8cb03ffe003aad349a/users/8f74db4e18c74cf593f2706020fcb6fb/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:08.980394 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69fc206-4ab5-43eb-bab5-ba7efc341aae admin admin] RBAC: Authorizing `identity:create_grant(project_id=8744e2ccdebc4c8cb03ffe003aad349a, user_id=8f74db4e18c74cf593f2706020fcb6fb, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:08.986238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69fc206-4ab5-43eb-bab5-ba7efc341aae admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:09.150105 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:09.157613 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1419/2852] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:49:08 2022] PUT /identity/v3/projects/8744e2ccdebc4c8cb03ffe003aad349a/users/8f74db4e18c74cf593f2706020fcb6fb/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:49:09.176323 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b5f859-894c-4059-9e45-1479f6418c6b None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:09.176745 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b5f859-894c-4059-9e45-1479f6418c6b None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:09.177143 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b5f859-894c-4059-9e45-1479f6418c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:09.216843 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-d7b5f859-894c-4059-9e45-1479f6418c6b None None] MFA Rules not processed for user `8f74db4e18c74cf593f2706020fcb6fb`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:09.218107 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-d7b5f859-894c-4059-9e45-1479f6418c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:09.294331 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1434/2853] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:49:09.301456 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69fc206-4ab5-43eb-bab5-ba7efc341aae admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:09.384739 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69fc206-4ab5-43eb-bab5-ba7efc341aae admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:09.390071 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d710a53-37e2-4dc7-bb41-df50a697c02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:09.390730 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d710a53-37e2-4dc7-bb41-df50a697c02a admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:09.391056 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d710a53-37e2-4dc7-bb41-df50a697c02a admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:09.391361 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d710a53-37e2-4dc7-bb41-df50a697c02a admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:09.392087 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d710a53-37e2-4dc7-bb41-df50a697c02a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:09.394043 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d710a53-37e2-4dc7-bb41-df50a697c02a admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:09.589554 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1420/2854] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:49:09 2022] POST /identity/v3/projects => generated 385 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:49:09.594477 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b5f859-894c-4059-9e45-1479f6418c6b None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:09.607209 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b5f859-894c-4059-9e45-1479f6418c6b None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:09.708153 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d7eb1-740e-4d05-8201-0e9839a502b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:09.708551 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d7eb1-740e-4d05-8201-0e9839a502b2 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:09.708927 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d7eb1-740e-4d05-8201-0e9839a502b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:09.709299 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d7eb1-740e-4d05-8201-0e9839a502b2 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:09.710056 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d7eb1-740e-4d05-8201-0e9839a502b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:09.712725 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d7eb1-740e-4d05-8201-0e9839a502b2 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:09.805891 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1435/2855] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:49:09 2022] POST /identity/v3/users => generated 299 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:49:09.810336 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d710a53-37e2-4dc7-bb41-df50a697c02a admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:09.825886 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d710a53-37e2-4dc7-bb41-df50a697c02a admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:09.831703 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a032c3f-d0ca-4b9b-b8da-bb2dfa810eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:09.832328 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a032c3f-d0ca-4b9b-b8da-bb2dfa810eaa admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:09.832606 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a032c3f-d0ca-4b9b-b8da-bb2dfa810eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:09.832873 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a032c3f-d0ca-4b9b-b8da-bb2dfa810eaa admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:09.833767 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a032c3f-d0ca-4b9b-b8da-bb2dfa810eaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:09.835308 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a032c3f-d0ca-4b9b-b8da-bb2dfa810eaa admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:09.850810 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1421/2856] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:49:09 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:49:09.856637 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d7eb1-740e-4d05-8201-0e9839a502b2 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:09.869092 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d7eb1-740e-4d05-8201-0e9839a502b2 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:09.874010 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bfb918-c66c-41bf-a414-1c552cd92073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:09.874749 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfb918-c66c-41bf-a414-1c552cd92073 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:09.875026 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfb918-c66c-41bf-a414-1c552cd92073 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:09.875368 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfb918-c66c-41bf-a414-1c552cd92073 admin admin] PATH_INFO: `/v3/projects/071b4cd3dfd94ba0988c115178b6f9f7/users/35efa9f2c9d44df98bccff207e96bcd8/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:09.895825 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bfb918-c66c-41bf-a414-1c552cd92073 admin admin] RBAC: Authorizing `identity:create_grant(project_id=071b4cd3dfd94ba0988c115178b6f9f7, user_id=35efa9f2c9d44df98bccff207e96bcd8, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:09.898845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bfb918-c66c-41bf-a414-1c552cd92073 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:10.007367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:10.014263 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1436/2857] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:49:09 2022] PUT /identity/v3/projects/071b4cd3dfd94ba0988c115178b6f9f7/users/35efa9f2c9d44df98bccff207e96bcd8/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:49:10.019925 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a032c3f-d0ca-4b9b-b8da-bb2dfa810eaa admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:10.032470 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a032c3f-d0ca-4b9b-b8da-bb2dfa810eaa admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:10.098463 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f7012d-97ee-40bb-92c4-c855f8ade38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:10.099005 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7012d-97ee-40bb-92c4-c855f8ade38f admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:10.099302 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7012d-97ee-40bb-92c4-c855f8ade38f admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:10.099646 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7012d-97ee-40bb-92c4-c855f8ade38f admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:10.100215 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f7012d-97ee-40bb-92c4-c855f8ade38f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:10.101837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f7012d-97ee-40bb-92c4-c855f8ade38f admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:10.113674 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1422/2858] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:49:10.118343 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bfb918-c66c-41bf-a414-1c552cd92073 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:10.130488 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bfb918-c66c-41bf-a414-1c552cd92073 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:10.138209 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe8f2a1-6086-429c-944c-3286c810ca03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:10.139392 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe8f2a1-6086-429c-944c-3286c810ca03 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:10.139918 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe8f2a1-6086-429c-944c-3286c810ca03 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:10.140387 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe8f2a1-6086-429c-944c-3286c810ca03 admin admin] PATH_INFO: `/v3/projects/071b4cd3dfd94ba0988c115178b6f9f7/users/35efa9f2c9d44df98bccff207e96bcd8/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:10.145419 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe8f2a1-6086-429c-944c-3286c810ca03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=071b4cd3dfd94ba0988c115178b6f9f7, user_id=35efa9f2c9d44df98bccff207e96bcd8, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:10.150791 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe8f2a1-6086-429c-944c-3286c810ca03 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:10.171288 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1437/2859] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:49:10 2022] PUT /identity/v3/projects/071b4cd3dfd94ba0988c115178b6f9f7/users/35efa9f2c9d44df98bccff207e96bcd8/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:49:10.189544 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ee4ea-0699-44fb-acf1-70e6cbec29f7 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:10.189917 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ee4ea-0699-44fb-acf1-70e6cbec29f7 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:10.190273 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ee4ea-0699-44fb-acf1-70e6cbec29f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:10.230171 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-777ee4ea-0699-44fb-acf1-70e6cbec29f7 None None] MFA Rules not processed for user `35efa9f2c9d44df98bccff207e96bcd8`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:10.231534 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-777ee4ea-0699-44fb-acf1-70e6cbec29f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:10.309316 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:10.320670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1423/2860] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:10 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:49:10.333666 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe8f2a1-6086-429c-944c-3286c810ca03 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:10.336009 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-9fe8f2a1-6086-429c-944c-3286c810ca03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:10.426265 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe8f2a1-6086-429c-944c-3286c810ca03 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:10.432564 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad7625-d89b-4d46-9401-db8509d74364 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:10.433616 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad7625-d89b-4d46-9401-db8509d74364 service cinder] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:10.434088 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad7625-d89b-4d46-9401-db8509d74364 service cinder] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:10.434844 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad7625-d89b-4d46-9401-db8509d74364 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:10.452483 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad7625-d89b-4d46-9401-db8509d74364 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:10.457025 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad7625-d89b-4d46-9401-db8509d74364 service cinder] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:10.477421 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1438/2861] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:49:10 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:49:10.559094 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ee4ea-0699-44fb-acf1-70e6cbec29f7 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:10.638311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ee4ea-0699-44fb-acf1-70e6cbec29f7 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:10.643079 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843650e-9cf1-40cf-9539-39e16c74b970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:10.643729 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843650e-9cf1-40cf-9539-39e16c74b970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:10.644004 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843650e-9cf1-40cf-9539-39e16c74b970 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:10.644275 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843650e-9cf1-40cf-9539-39e16c74b970 admin admin] PATH_INFO: `/v3/users/8f74db4e18c74cf593f2706020fcb6fb` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:10.645473 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843650e-9cf1-40cf-9539-39e16c74b970 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f74db4e18c74cf593f2706020fcb6fb)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:10.647462 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843650e-9cf1-40cf-9539-39e16c74b970 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:10.833977 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-0843650e-9cf1-40cf-9539-39e16c74b970 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f74db4e18c74cf593f2706020fcb6fb'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:10.911202 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-0843650e-9cf1-40cf-9539-39e16c74b970 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f74db4e18c74cf593f2706020fcb6fb'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:10.911788 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-0843650e-9cf1-40cf-9539-39e16c74b970 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f74db4e18c74cf593f2706020fcb6fb'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:10.912719 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-0843650e-9cf1-40cf-9539-39e16c74b970 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f74db4e18c74cf593f2706020fcb6fb'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:10.935624 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-0843650e-9cf1-40cf-9539-39e16c74b970 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f74db4e18c74cf593f2706020fcb6fb'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:10.959939 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1424/2862] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:49:10 2022] DELETE /identity/v3/users/8f74db4e18c74cf593f2706020fcb6fb => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:49:10.966467 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435830f-e881-4dd3-95d7-3a0240a3012e None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:10.966979 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435830f-e881-4dd3-95d7-3a0240a3012e None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:10.967100 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435830f-e881-4dd3-95d7-3a0240a3012e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:10.987453 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-7435830f-e881-4dd3-95d7-3a0240a3012e None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:10.988727 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-7435830f-e881-4dd3-95d7-3a0240a3012e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:11.056978 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1439/2863] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:49:11.068024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843650e-9cf1-40cf-9539-39e16c74b970 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:11.069790 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-0843650e-9cf1-40cf-9539-39e16c74b970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:11.142986 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843650e-9cf1-40cf-9539-39e16c74b970 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:11.146989 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21bc05-1337-474a-a112-4731eefe0a22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:11.148251 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21bc05-1337-474a-a112-4731eefe0a22 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:11.148251 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21bc05-1337-474a-a112-4731eefe0a22 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:11.148251 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21bc05-1337-474a-a112-4731eefe0a22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:11.160153 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21bc05-1337-474a-a112-4731eefe0a22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:11.162920 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21bc05-1337-474a-a112-4731eefe0a22 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:11.179416 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1425/2864] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:49:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:49:11.760098 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435830f-e881-4dd3-95d7-3a0240a3012e None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:11.760861 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-7435830f-e881-4dd3-95d7-3a0240a3012e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:11.778587 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435830f-e881-4dd3-95d7-3a0240a3012e None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:11.785167 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26c1dd-ffad-4d3d-a05b-2dc886ec3a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:11.785930 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26c1dd-ffad-4d3d-a05b-2dc886ec3a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:11.786271 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26c1dd-ffad-4d3d-a05b-2dc886ec3a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:11.786647 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26c1dd-ffad-4d3d-a05b-2dc886ec3a8c admin admin] PATH_INFO: `/v3/projects/8744e2ccdebc4c8cb03ffe003aad349a` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:11.787977 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb26c1dd-ffad-4d3d-a05b-2dc886ec3a8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8744e2ccdebc4c8cb03ffe003aad349a)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:11.790288 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb26c1dd-ffad-4d3d-a05b-2dc886ec3a8c admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:11.948843 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cb26c1dd-ffad-4d3d-a05b-2dc886ec3a8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8744e2ccdebc4c8cb03ffe003aad349a', 'request_id': 'req-cb26c1dd-ffad-4d3d-a05b-2dc886ec3a8c'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:12.043014 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cb26c1dd-ffad-4d3d-a05b-2dc886ec3a8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8744e2ccdebc4c8cb03ffe003aad349a', 'request_id': 'req-cb26c1dd-ffad-4d3d-a05b-2dc886ec3a8c'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:12.057749 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cb26c1dd-ffad-4d3d-a05b-2dc886ec3a8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8744e2ccdebc4c8cb03ffe003aad349a', 'request_id': 'req-cb26c1dd-ffad-4d3d-a05b-2dc886ec3a8c'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:12.078193 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cb26c1dd-ffad-4d3d-a05b-2dc886ec3a8c admin admin] The token cache is being invalidate because project 8744e2ccdebc4c8cb03ffe003aad349a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:49:12.081921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cb26c1dd-ffad-4d3d-a05b-2dc886ec3a8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:12.081921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-cb26c1dd-ffad-4d3d-a05b-2dc886ec3a8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:12.081921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1440/2865] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:49:11 2022] DELETE /identity/v3/projects/8744e2ccdebc4c8cb03ffe003aad349a => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:49:12.086166 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21bc05-1337-474a-a112-4731eefe0a22 service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:12.099884 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21bc05-1337-474a-a112-4731eefe0a22 service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:12.120451 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:12.178946 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d6437-6368-4d7c-87bf-3c59112415ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:12.179730 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d6437-6368-4d7c-87bf-3c59112415ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:12.180100 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d6437-6368-4d7c-87bf-3c59112415ee admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:12.180472 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d6437-6368-4d7c-87bf-3c59112415ee admin admin] PATH_INFO: `/v3/users/35efa9f2c9d44df98bccff207e96bcd8` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:12.181851 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d6437-6368-4d7c-87bf-3c59112415ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=35efa9f2c9d44df98bccff207e96bcd8)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:12.184171 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d6437-6368-4d7c-87bf-3c59112415ee admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:12.435076 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-b41d6437-6368-4d7c-87bf-3c59112415ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35efa9f2c9d44df98bccff207e96bcd8'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:12.503508 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-b41d6437-6368-4d7c-87bf-3c59112415ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35efa9f2c9d44df98bccff207e96bcd8'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:12.504292 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-b41d6437-6368-4d7c-87bf-3c59112415ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35efa9f2c9d44df98bccff207e96bcd8'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:12.505171 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-b41d6437-6368-4d7c-87bf-3c59112415ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35efa9f2c9d44df98bccff207e96bcd8'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:12.525691 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-b41d6437-6368-4d7c-87bf-3c59112415ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35efa9f2c9d44df98bccff207e96bcd8'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:12.549148 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1426/2866] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:49:12 2022] DELETE /identity/v3/users/35efa9f2c9d44df98bccff207e96bcd8 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:49:13.054118 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26c1dd-ffad-4d3d-a05b-2dc886ec3a8c admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:13.055788 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-cb26c1dd-ffad-4d3d-a05b-2dc886ec3a8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:13.079078 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26c1dd-ffad-4d3d-a05b-2dc886ec3a8c admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:13.090359 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:13.148921 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fab66-5624-4056-9b61-28fcd2a1d2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:13.150351 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fab66-5624-4056-9b61-28fcd2a1d2a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:13.150887 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fab66-5624-4056-9b61-28fcd2a1d2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:13.151378 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fab66-5624-4056-9b61-28fcd2a1d2a7 admin admin] PATH_INFO: `/v3/projects/071b4cd3dfd94ba0988c115178b6f9f7` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:13.153432 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245fab66-5624-4056-9b61-28fcd2a1d2a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=071b4cd3dfd94ba0988c115178b6f9f7)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:13.157287 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245fab66-5624-4056-9b61-28fcd2a1d2a7 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:13.284704 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-245fab66-5624-4056-9b61-28fcd2a1d2a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '071b4cd3dfd94ba0988c115178b6f9f7', 'request_id': 'req-245fab66-5624-4056-9b61-28fcd2a1d2a7'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:13.374488 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-245fab66-5624-4056-9b61-28fcd2a1d2a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '071b4cd3dfd94ba0988c115178b6f9f7', 'request_id': 'req-245fab66-5624-4056-9b61-28fcd2a1d2a7'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:13.382546 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-245fab66-5624-4056-9b61-28fcd2a1d2a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '071b4cd3dfd94ba0988c115178b6f9f7', 'request_id': 'req-245fab66-5624-4056-9b61-28fcd2a1d2a7'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:13.396045 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-245fab66-5624-4056-9b61-28fcd2a1d2a7 admin admin] The token cache is being invalidate because project 071b4cd3dfd94ba0988c115178b6f9f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:49:13.396355 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-245fab66-5624-4056-9b61-28fcd2a1d2a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:13.397025 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-245fab66-5624-4056-9b61-28fcd2a1d2a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:13.398607 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1441/2867] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:49:13 2022] DELETE /identity/v3/projects/071b4cd3dfd94ba0988c115178b6f9f7 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:49:13.403902 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c248788-3baf-4b6d-ad66-6a8255a301ae None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:13.404468 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c248788-3baf-4b6d-ad66-6a8255a301ae None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:13.404468 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c248788-3baf-4b6d-ad66-6a8255a301ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:13.426760 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-2c248788-3baf-4b6d-ad66-6a8255a301ae None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:13.428156 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-2c248788-3baf-4b6d-ad66-6a8255a301ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:13.503343 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1427/2868] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:49:13.512281 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde528e2-d8c2-401b-81d1-cb5ed9ef0bfd None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:13.512563 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde528e2-d8c2-401b-81d1-cb5ed9ef0bfd None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:13.512840 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde528e2-d8c2-401b-81d1-cb5ed9ef0bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:13.533561 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-bde528e2-d8c2-401b-81d1-cb5ed9ef0bfd None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:13.534835 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-bde528e2-d8c2-401b-81d1-cb5ed9ef0bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:13.543162 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1442/2869] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:49:13.548410 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c248788-3baf-4b6d-ad66-6a8255a301ae None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:13.559607 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c248788-3baf-4b6d-ad66-6a8255a301ae None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:13.565313 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3707083f-e074-494f-bf1f-2c21af344cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:13.565937 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3707083f-e074-494f-bf1f-2c21af344cc4 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:13.566219 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3707083f-e074-494f-bf1f-2c21af344cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:13.566527 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3707083f-e074-494f-bf1f-2c21af344cc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:13.567193 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3707083f-e074-494f-bf1f-2c21af344cc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:49:13.567498 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3707083f-e074-494f-bf1f-2c21af344cc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:13.569120 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3707083f-e074-494f-bf1f-2c21af344cc4 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:13.582693 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1428/2870] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:49:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:49:13.588003 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde528e2-d8c2-401b-81d1-cb5ed9ef0bfd None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:13.598263 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde528e2-d8c2-401b-81d1-cb5ed9ef0bfd None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:13.603250 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d1a96a-6911-423d-b1e6-0a2fc3aae292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:13.603874 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1a96a-6911-423d-b1e6-0a2fc3aae292 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:13.604154 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1a96a-6911-423d-b1e6-0a2fc3aae292 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:13.604427 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1a96a-6911-423d-b1e6-0a2fc3aae292 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:13.605078 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d1a96a-6911-423d-b1e6-0a2fc3aae292 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:13.607102 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d1a96a-6911-423d-b1e6-0a2fc3aae292 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:13.701226 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1443/2871] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:49:13 2022] POST /identity/v3/projects => generated 389 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:49:13.706238 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3707083f-e074-494f-bf1f-2c21af344cc4 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:13.717440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3707083f-e074-494f-bf1f-2c21af344cc4 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:13.743989 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:13.785230 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a00c5-8c8c-4984-b1cc-57b918591af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:13.785836 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a00c5-8c8c-4984-b1cc-57b918591af9 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:13.786110 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a00c5-8c8c-4984-b1cc-57b918591af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:13.786376 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a00c5-8c8c-4984-b1cc-57b918591af9 admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:13.787044 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a00c5-8c8c-4984-b1cc-57b918591af9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:13.789026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a00c5-8c8c-4984-b1cc-57b918591af9 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:13.904667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1429/2872] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:49:13 2022] POST /identity/v3/users => generated 295 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:49:13.908981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d1a96a-6911-423d-b1e6-0a2fc3aae292 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:13.920320 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d1a96a-6911-423d-b1e6-0a2fc3aae292 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:13.927795 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a1a7e-0031-4c25-92cc-3a0b283615e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:13.928211 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a1a7e-0031-4c25-92cc-3a0b283615e7 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:13.928661 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a1a7e-0031-4c25-92cc-3a0b283615e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:13.929084 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a1a7e-0031-4c25-92cc-3a0b283615e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:13.929782 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a1a7e-0031-4c25-92cc-3a0b283615e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:13.932401 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a1a7e-0031-4c25-92cc-3a0b283615e7 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:13.946447 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1444/2873] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:49:13.950728 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a00c5-8c8c-4984-b1cc-57b918591af9 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:13.962558 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a00c5-8c8c-4984-b1cc-57b918591af9 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:13.967594 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca20ac78-5f03-47d4-b0ca-43d5c720b1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:13.968219 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20ac78-5f03-47d4-b0ca-43d5c720b1ee admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:13.968491 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20ac78-5f03-47d4-b0ca-43d5c720b1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:13.968757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20ac78-5f03-47d4-b0ca-43d5c720b1ee admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:13.969926 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca20ac78-5f03-47d4-b0ca-43d5c720b1ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:13.971563 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca20ac78-5f03-47d4-b0ca-43d5c720b1ee admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:13.982765 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1430/2874] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:49:13.990786 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a1a7e-0031-4c25-92cc-3a0b283615e7 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:14.002231 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a1a7e-0031-4c25-92cc-3a0b283615e7 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:14.007312 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2fe222-4162-414f-bbb5-c276c91f2abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:14.007985 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fe222-4162-414f-bbb5-c276c91f2abb admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:14.008264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fe222-4162-414f-bbb5-c276c91f2abb admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:14.008540 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fe222-4162-414f-bbb5-c276c91f2abb admin admin] PATH_INFO: `/v3/projects/3665df0e81484e8382cf2bc4105cb63c/users/1ee2f3c2c2ef4500a930a2637b425f88/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:14.030881 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2fe222-4162-414f-bbb5-c276c91f2abb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3665df0e81484e8382cf2bc4105cb63c, user_id=1ee2f3c2c2ef4500a930a2637b425f88, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:14.034404 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2fe222-4162-414f-bbb5-c276c91f2abb admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:14.116720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1445/2875] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:49:13 2022] PUT /identity/v3/projects/3665df0e81484e8382cf2bc4105cb63c/users/1ee2f3c2c2ef4500a930a2637b425f88/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:49:14.130733 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9164a-f51a-4d69-840f-75a6fb89df07 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:14.133038 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9164a-f51a-4d69-840f-75a6fb89df07 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:14.133038 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9164a-f51a-4d69-840f-75a6fb89df07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:14.165161 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-9af9164a-f51a-4d69-840f-75a6fb89df07 None None] MFA Rules not processed for user `1ee2f3c2c2ef4500a930a2637b425f88`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:14.166430 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-9af9164a-f51a-4d69-840f-75a6fb89df07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:14.242495 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1431/2876] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:49:14.247184 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2fe222-4162-414f-bbb5-c276c91f2abb admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:14.295430 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:14.324685 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2fe222-4162-414f-bbb5-c276c91f2abb admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:14.332833 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2061d25-eaff-42d7-8a28-955ec08ab161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:14.333699 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2061d25-eaff-42d7-8a28-955ec08ab161 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:14.334433 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2061d25-eaff-42d7-8a28-955ec08ab161 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:14.334636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2061d25-eaff-42d7-8a28-955ec08ab161 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:14.335461 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2061d25-eaff-42d7-8a28-955ec08ab161 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:14.338282 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2061d25-eaff-42d7-8a28-955ec08ab161 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:14.448434 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1446/2877] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:49:14 2022] POST /identity/v3/projects => generated 385 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:49:14.450771 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af9164a-f51a-4d69-840f-75a6fb89df07 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:14.464322 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af9164a-f51a-4d69-840f-75a6fb89df07 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:14.532178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c1c27-fe59-4510-bb8b-679ae65a11a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:14.532775 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c1c27-fe59-4510-bb8b-679ae65a11a6 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:14.533131 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c1c27-fe59-4510-bb8b-679ae65a11a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:14.533413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c1c27-fe59-4510-bb8b-679ae65a11a6 admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:14.534073 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27c1c27-fe59-4510-bb8b-679ae65a11a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:14.536143 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27c1c27-fe59-4510-bb8b-679ae65a11a6 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:14.654651 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1432/2878] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:49:14 2022] POST /identity/v3/users => generated 299 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:49:14.659833 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2061d25-eaff-42d7-8a28-955ec08ab161 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:14.671134 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2061d25-eaff-42d7-8a28-955ec08ab161 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:14.677439 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e490f738-d6f6-4ecf-98d8-6b2e02aa7e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:14.678058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490f738-d6f6-4ecf-98d8-6b2e02aa7e78 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:14.678333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490f738-d6f6-4ecf-98d8-6b2e02aa7e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:14.678639 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490f738-d6f6-4ecf-98d8-6b2e02aa7e78 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:14.679302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e490f738-d6f6-4ecf-98d8-6b2e02aa7e78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:14.680932 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e490f738-d6f6-4ecf-98d8-6b2e02aa7e78 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:14.693152 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1447/2879] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:49:14.697453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c1c27-fe59-4510-bb8b-679ae65a11a6 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:14.709137 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c1c27-fe59-4510-bb8b-679ae65a11a6 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:14.714020 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e8e6d-988c-42c5-8c63-24e59a996b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:14.714723 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e8e6d-988c-42c5-8c63-24e59a996b73 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:14.714992 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e8e6d-988c-42c5-8c63-24e59a996b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:14.715271 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e8e6d-988c-42c5-8c63-24e59a996b73 admin admin] PATH_INFO: `/v3/projects/b0e17aa1ccb04901873ea83657277ce3/users/d9955ad21917430aacdf39847782aa8e/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:14.758908 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e8e6d-988c-42c5-8c63-24e59a996b73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0e17aa1ccb04901873ea83657277ce3, user_id=d9955ad21917430aacdf39847782aa8e, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:14.764433 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e8e6d-988c-42c5-8c63-24e59a996b73 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:14.889744 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:14.896631 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1433/2880] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:49:14 2022] PUT /identity/v3/projects/b0e17aa1ccb04901873ea83657277ce3/users/d9955ad21917430aacdf39847782aa8e/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:49:14.901620 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e490f738-d6f6-4ecf-98d8-6b2e02aa7e78 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:14.917952 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e490f738-d6f6-4ecf-98d8-6b2e02aa7e78 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:14.983518 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9abfaa-8198-4854-b09e-95d35a749dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:14.984128 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9abfaa-8198-4854-b09e-95d35a749dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:14.984409 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9abfaa-8198-4854-b09e-95d35a749dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:14.984672 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9abfaa-8198-4854-b09e-95d35a749dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:14.985340 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9abfaa-8198-4854-b09e-95d35a749dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:14.986992 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9abfaa-8198-4854-b09e-95d35a749dd4 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:14.997744 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1448/2881] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:49:15.003425 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e8e6d-988c-42c5-8c63-24e59a996b73 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:15.014678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e8e6d-988c-42c5-8c63-24e59a996b73 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:15.020269 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118dcec-b844-46d8-85cf-7d55c1790a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:15.020928 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118dcec-b844-46d8-85cf-7d55c1790a71 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:15.021210 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118dcec-b844-46d8-85cf-7d55c1790a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:15.021481 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118dcec-b844-46d8-85cf-7d55c1790a71 admin admin] PATH_INFO: `/v3/projects/b0e17aa1ccb04901873ea83657277ce3/users/d9955ad21917430aacdf39847782aa8e/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:15.023508 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b118dcec-b844-46d8-85cf-7d55c1790a71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0e17aa1ccb04901873ea83657277ce3, user_id=d9955ad21917430aacdf39847782aa8e, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:15.026243 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b118dcec-b844-46d8-85cf-7d55c1790a71 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:15.039584 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1434/2882] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:49:15 2022] PUT /identity/v3/projects/b0e17aa1ccb04901873ea83657277ce3/users/d9955ad21917430aacdf39847782aa8e/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:49:15.050941 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c8712-44b8-4ece-aa8b-75d198b93e69 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:15.054286 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c8712-44b8-4ece-aa8b-75d198b93e69 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:15.054286 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c8712-44b8-4ece-aa8b-75d198b93e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:15.099969 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-c37c8712-44b8-4ece-aa8b-75d198b93e69 None None] MFA Rules not processed for user `d9955ad21917430aacdf39847782aa8e`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:15.102009 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-c37c8712-44b8-4ece-aa8b-75d198b93e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:15.228577 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1449/2883] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:15 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:49:15.243190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118dcec-b844-46d8-85cf-7d55c1790a71 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:15.243664 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-b118dcec-b844-46d8-85cf-7d55c1790a71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:15.327388 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118dcec-b844-46d8-85cf-7d55c1790a71 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:15.334559 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50358811-492f-4ae3-adc4-bccf67ef6427 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:15.335634 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50358811-492f-4ae3-adc4-bccf67ef6427 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:15.336132 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50358811-492f-4ae3-adc4-bccf67ef6427 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:15.336627 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50358811-492f-4ae3-adc4-bccf67ef6427 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:15.349406 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50358811-492f-4ae3-adc4-bccf67ef6427 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:15.351999 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50358811-492f-4ae3-adc4-bccf67ef6427 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:15.369649 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1435/2884] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:49:15 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:49:15.562017 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c8712-44b8-4ece-aa8b-75d198b93e69 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:15.599066 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50358811-492f-4ae3-adc4-bccf67ef6427 service cinder] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:15.601478 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-50358811-492f-4ae3-adc4-bccf67ef6427 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:15.610300 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:15.648592 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c8712-44b8-4ece-aa8b-75d198b93e69 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:15.653971 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348595e-1154-485c-b43f-79e124ff67bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:15.654822 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348595e-1154-485c-b43f-79e124ff67bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:15.655459 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348595e-1154-485c-b43f-79e124ff67bf admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:15.655953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348595e-1154-485c-b43f-79e124ff67bf admin admin] PATH_INFO: `/v3/users/1ee2f3c2c2ef4500a930a2637b425f88` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:15.658549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2348595e-1154-485c-b43f-79e124ff67bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ee2f3c2c2ef4500a930a2637b425f88)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:15.662412 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2348595e-1154-485c-b43f-79e124ff67bf admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:15.676070 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50358811-492f-4ae3-adc4-bccf67ef6427 service cinder] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:15.680589 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada3d09-5af4-481d-a0a2-34a593b50785 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:15.681251 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada3d09-5af4-481d-a0a2-34a593b50785 service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:15.681514 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada3d09-5af4-481d-a0a2-34a593b50785 service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:15.681781 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada3d09-5af4-481d-a0a2-34a593b50785 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:15.683663 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-9ada3d09-5af4-481d-a0a2-34a593b50785 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:15.760292 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada3d09-5af4-481d-a0a2-34a593b50785 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:15.761799 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada3d09-5af4-481d-a0a2-34a593b50785 service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:15.772555 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1436/2885] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:49:15 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:49:15.962115 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-2348595e-1154-485c-b43f-79e124ff67bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ee2f3c2c2ef4500a930a2637b425f88'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:15.976959 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-2348595e-1154-485c-b43f-79e124ff67bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ee2f3c2c2ef4500a930a2637b425f88'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:15.985802 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-2348595e-1154-485c-b43f-79e124ff67bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ee2f3c2c2ef4500a930a2637b425f88'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:16.075386 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-2348595e-1154-485c-b43f-79e124ff67bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ee2f3c2c2ef4500a930a2637b425f88'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:16.076769 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-2348595e-1154-485c-b43f-79e124ff67bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ee2f3c2c2ef4500a930a2637b425f88'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:16.087618 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1450/2886] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:49:15 2022] DELETE /identity/v3/users/1ee2f3c2c2ef4500a930a2637b425f88 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:49:16.095303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd396d-9f0a-42ea-b805-484c494f8edb None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:16.095730 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd396d-9f0a-42ea-b805-484c494f8edb None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:16.096145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd396d-9f0a-42ea-b805-484c494f8edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:16.123786 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-21cd396d-9f0a-42ea-b805-484c494f8edb None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:16.125474 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-21cd396d-9f0a-42ea-b805-484c494f8edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:16.166829 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:16.201453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1437/2887] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:49:16.227276 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348595e-1154-485c-b43f-79e124ff67bf admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:16.229221 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2348595e-1154-485c-b43f-79e124ff67bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:16.337315 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348595e-1154-485c-b43f-79e124ff67bf admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:16.345221 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0bbd3-10e4-45f1-8210-5f4d986a8761 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:16.346069 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0bbd3-10e4-45f1-8210-5f4d986a8761 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:16.346569 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0bbd3-10e4-45f1-8210-5f4d986a8761 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:16.347061 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0bbd3-10e4-45f1-8210-5f4d986a8761 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:16.364902 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0bbd3-10e4-45f1-8210-5f4d986a8761 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:16.367756 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0bbd3-10e4-45f1-8210-5f4d986a8761 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:16.387253 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1451/2888] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:49:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:49:16.932906 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd396d-9f0a-42ea-b805-484c494f8edb None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:16.935439 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-21cd396d-9f0a-42ea-b805-484c494f8edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:16.950024 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd396d-9f0a-42ea-b805-484c494f8edb None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:16.956805 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54561c47-4f48-4d3b-98e8-90b31ded9ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:16.957473 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54561c47-4f48-4d3b-98e8-90b31ded9ba4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:16.957761 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54561c47-4f48-4d3b-98e8-90b31ded9ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:16.958033 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54561c47-4f48-4d3b-98e8-90b31ded9ba4 admin admin] PATH_INFO: `/v3/projects/3665df0e81484e8382cf2bc4105cb63c` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:16.959418 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54561c47-4f48-4d3b-98e8-90b31ded9ba4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3665df0e81484e8382cf2bc4105cb63c)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:16.961599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54561c47-4f48-4d3b-98e8-90b31ded9ba4 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:17.110754 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-54561c47-4f48-4d3b-98e8-90b31ded9ba4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3665df0e81484e8382cf2bc4105cb63c', 'request_id': 'req-54561c47-4f48-4d3b-98e8-90b31ded9ba4'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:17.118383 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-54561c47-4f48-4d3b-98e8-90b31ded9ba4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3665df0e81484e8382cf2bc4105cb63c', 'request_id': 'req-54561c47-4f48-4d3b-98e8-90b31ded9ba4'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:17.127628 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-54561c47-4f48-4d3b-98e8-90b31ded9ba4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3665df0e81484e8382cf2bc4105cb63c', 'request_id': 'req-54561c47-4f48-4d3b-98e8-90b31ded9ba4'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:17.236246 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-54561c47-4f48-4d3b-98e8-90b31ded9ba4 admin admin] The token cache is being invalidate because project 3665df0e81484e8382cf2bc4105cb63c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:49:17.236615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-54561c47-4f48-4d3b-98e8-90b31ded9ba4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:17.237250 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-54561c47-4f48-4d3b-98e8-90b31ded9ba4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:17.238894 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1438/2889] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:49:16 2022] DELETE /identity/v3/projects/3665df0e81484e8382cf2bc4105cb63c => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:49:17.243246 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0bbd3-10e4-45f1-8210-5f4d986a8761 service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:17.256048 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0bbd3-10e4-45f1-8210-5f4d986a8761 service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:17.335911 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:17.346535 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689874ec-dc7b-4e69-9795-c5ad582b3f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:17.347305 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689874ec-dc7b-4e69-9795-c5ad582b3f20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:17.347588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689874ec-dc7b-4e69-9795-c5ad582b3f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:17.347861 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689874ec-dc7b-4e69-9795-c5ad582b3f20 admin admin] PATH_INFO: `/v3/users/d9955ad21917430aacdf39847782aa8e` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:17.349468 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689874ec-dc7b-4e69-9795-c5ad582b3f20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9955ad21917430aacdf39847782aa8e)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:17.351977 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689874ec-dc7b-4e69-9795-c5ad582b3f20 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:17.655271 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-689874ec-dc7b-4e69-9795-c5ad582b3f20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9955ad21917430aacdf39847782aa8e'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:17.670670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-689874ec-dc7b-4e69-9795-c5ad582b3f20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9955ad21917430aacdf39847782aa8e'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:17.678298 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-689874ec-dc7b-4e69-9795-c5ad582b3f20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9955ad21917430aacdf39847782aa8e'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:17.762214 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-689874ec-dc7b-4e69-9795-c5ad582b3f20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9955ad21917430aacdf39847782aa8e'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:17.763004 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-689874ec-dc7b-4e69-9795-c5ad582b3f20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9955ad21917430aacdf39847782aa8e'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:17.771558 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1452/2890] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:49:17 2022] DELETE /identity/v3/users/d9955ad21917430aacdf39847782aa8e => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:49:18.364911 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54561c47-4f48-4d3b-98e8-90b31ded9ba4 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:18.367185 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-54561c47-4f48-4d3b-98e8-90b31ded9ba4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:18.391244 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54561c47-4f48-4d3b-98e8-90b31ded9ba4 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:18.477319 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b66def-a88f-470d-aaa8-b0f4991f3c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:18.478121 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b66def-a88f-470d-aaa8-b0f4991f3c48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:18.478559 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b66def-a88f-470d-aaa8-b0f4991f3c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:18.478960 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b66def-a88f-470d-aaa8-b0f4991f3c48 admin admin] PATH_INFO: `/v3/projects/b0e17aa1ccb04901873ea83657277ce3` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:18.480365 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b66def-a88f-470d-aaa8-b0f4991f3c48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0e17aa1ccb04901873ea83657277ce3)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:18.482350 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b66def-a88f-470d-aaa8-b0f4991f3c48 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:18.565333 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689874ec-dc7b-4e69-9795-c5ad582b3f20 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:18.567057 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-689874ec-dc7b-4e69-9795-c5ad582b3f20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:18.631311 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-c6b66def-a88f-470d-aaa8-b0f4991f3c48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0e17aa1ccb04901873ea83657277ce3', 'request_id': 'req-c6b66def-a88f-470d-aaa8-b0f4991f3c48'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:18.640123 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-c6b66def-a88f-470d-aaa8-b0f4991f3c48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0e17aa1ccb04901873ea83657277ce3', 'request_id': 'req-c6b66def-a88f-470d-aaa8-b0f4991f3c48'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:18.649933 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-c6b66def-a88f-470d-aaa8-b0f4991f3c48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0e17aa1ccb04901873ea83657277ce3', 'request_id': 'req-c6b66def-a88f-470d-aaa8-b0f4991f3c48'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:18.652095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:18.688549 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689874ec-dc7b-4e69-9795-c5ad582b3f20 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:18.733645 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-c6b66def-a88f-470d-aaa8-b0f4991f3c48 admin admin] The token cache is being invalidate because project b0e17aa1ccb04901873ea83657277ce3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:49:18.734162 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-c6b66def-a88f-470d-aaa8-b0f4991f3c48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:18.735493 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-c6b66def-a88f-470d-aaa8-b0f4991f3c48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:18.741096 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1439/2891] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:49:18 2022] DELETE /identity/v3/projects/b0e17aa1ccb04901873ea83657277ce3 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:49:18.747686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185314e-496c-43bd-8a56-3de039c1e4d1 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:18.747686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185314e-496c-43bd-8a56-3de039c1e4d1 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:18.747686 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185314e-496c-43bd-8a56-3de039c1e4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:18.770194 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-e185314e-496c-43bd-8a56-3de039c1e4d1 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:18.771920 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-e185314e-496c-43bd-8a56-3de039c1e4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:18.800976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e57b6-d035-4253-8bb3-e85457dd2e16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:18.802119 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e57b6-d035-4253-8bb3-e85457dd2e16 service nova] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:18.802978 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e57b6-d035-4253-8bb3-e85457dd2e16 service nova] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:18.803191 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e57b6-d035-4253-8bb3-e85457dd2e16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:18.807172 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-9b5e57b6-d035-4253-8bb3-e85457dd2e16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:18.830310 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e57b6-d035-4253-8bb3-e85457dd2e16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:18.834643 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e57b6-d035-4253-8bb3-e85457dd2e16 service nova] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:18.841613 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:18.853309 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1453/2892] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:49:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:49:18.868344 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1440/2893] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:49:18.878900 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fe3a2-c219-4f0a-b276-011642827772 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:18.879509 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fe3a2-c219-4f0a-b276-011642827772 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:18.880127 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fe3a2-c219-4f0a-b276-011642827772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:18.921303 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-fc5fe3a2-c219-4f0a-b276-011642827772 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:18.924060 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-fc5fe3a2-c219-4f0a-b276-011642827772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:18.934074 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1454/2894] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:49:18.939293 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e185314e-496c-43bd-8a56-3de039c1e4d1 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:18.951126 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e185314e-496c-43bd-8a56-3de039c1e4d1 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:18.958301 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36082179-b507-4d5c-9d1c-6bb68845963a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:18.965148 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36082179-b507-4d5c-9d1c-6bb68845963a admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:18.965148 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36082179-b507-4d5c-9d1c-6bb68845963a admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:18.965148 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36082179-b507-4d5c-9d1c-6bb68845963a admin admin] PATH_INFO: `/v3/domains` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:18.965148 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36082179-b507-4d5c-9d1c-6bb68845963a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82089) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:49:18.965148 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36082179-b507-4d5c-9d1c-6bb68845963a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:18.965148 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36082179-b507-4d5c-9d1c-6bb68845963a admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:18.990172 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1441/2895] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:49:18.995203 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5fe3a2-c219-4f0a-b276-011642827772 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:19.009187 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5fe3a2-c219-4f0a-b276-011642827772 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:19.017694 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12ca07-7c46-49a2-8f89-060155db2c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:19.018344 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12ca07-7c46-49a2-8f89-060155db2c91 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:19.018683 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12ca07-7c46-49a2-8f89-060155db2c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:19.018956 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12ca07-7c46-49a2-8f89-060155db2c91 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:19.019630 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12ca07-7c46-49a2-8f89-060155db2c91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:19.021598 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12ca07-7c46-49a2-8f89-060155db2c91 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:19.153945 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1455/2896] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:49:18 2022] POST /identity/v3/projects => generated 381 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:49:19.158870 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36082179-b507-4d5c-9d1c-6bb68845963a admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:19.172616 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36082179-b507-4d5c-9d1c-6bb68845963a admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:19.251101 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b0e69-a163-468e-b3c2-6b05ff713d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:19.251101 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b0e69-a163-468e-b3c2-6b05ff713d2b admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:19.251393 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b0e69-a163-468e-b3c2-6b05ff713d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:19.251599 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b0e69-a163-468e-b3c2-6b05ff713d2b admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:19.252446 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01b0e69-a163-468e-b3c2-6b05ff713d2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:19.254558 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01b0e69-a163-468e-b3c2-6b05ff713d2b admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:19.378288 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1442/2897] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:49:19 2022] POST /identity/v3/users => generated 291 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:49:19.384642 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12ca07-7c46-49a2-8f89-060155db2c91 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:19.405141 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12ca07-7c46-49a2-8f89-060155db2c91 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:19.414543 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e78f0-f91e-4046-9d5e-cd6ca43afb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:19.415675 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e78f0-f91e-4046-9d5e-cd6ca43afb9f admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:19.416169 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e78f0-f91e-4046-9d5e-cd6ca43afb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:19.417546 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e78f0-f91e-4046-9d5e-cd6ca43afb9f admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:19.417931 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e78f0-f91e-4046-9d5e-cd6ca43afb9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:19.420968 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e78f0-f91e-4046-9d5e-cd6ca43afb9f admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:19.443823 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1456/2898] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:49:19.448718 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b0e69-a163-468e-b3c2-6b05ff713d2b admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:19.463443 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b0e69-a163-468e-b3c2-6b05ff713d2b admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:19.472502 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf36f54-346b-4dcf-b360-50ffab34a192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:19.473605 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf36f54-346b-4dcf-b360-50ffab34a192 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:19.474103 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf36f54-346b-4dcf-b360-50ffab34a192 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:19.474647 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf36f54-346b-4dcf-b360-50ffab34a192 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:19.475828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf36f54-346b-4dcf-b360-50ffab34a192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:19.479113 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf36f54-346b-4dcf-b360-50ffab34a192 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:19.502586 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1443/2899] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:49:19.506677 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e78f0-f91e-4046-9d5e-cd6ca43afb9f admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:19.517854 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e78f0-f91e-4046-9d5e-cd6ca43afb9f admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:19.523149 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc672d25-6870-41f4-99c5-a7b2e4b0a200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:19.523907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc672d25-6870-41f4-99c5-a7b2e4b0a200 admin admin] REQUEST_METHOD: `PUT` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:19.524190 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc672d25-6870-41f4-99c5-a7b2e4b0a200 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:19.524461 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc672d25-6870-41f4-99c5-a7b2e4b0a200 admin admin] PATH_INFO: `/v3/projects/18df80ee69704ccda6e8a9c9cde627b0/users/3f958f45da7a499c9d9b8d6af4a39cc0/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:19.546307 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc672d25-6870-41f4-99c5-a7b2e4b0a200 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18df80ee69704ccda6e8a9c9cde627b0, user_id=3f958f45da7a499c9d9b8d6af4a39cc0, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:19.548799 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc672d25-6870-41f4-99c5-a7b2e4b0a200 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:19.632724 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1457/2900] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:49:19 2022] PUT /identity/v3/projects/18df80ee69704ccda6e8a9c9cde627b0/users/3f958f45da7a499c9d9b8d6af4a39cc0/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:49:19.652959 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a357df3-2fa7-4998-833c-3c70909cd0b5 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:19.653451 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a357df3-2fa7-4998-833c-3c70909cd0b5 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:19.653931 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a357df3-2fa7-4998-833c-3c70909cd0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:19.706838 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-3a357df3-2fa7-4998-833c-3c70909cd0b5 None None] MFA Rules not processed for user `3f958f45da7a499c9d9b8d6af4a39cc0`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:19.709100 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-3a357df3-2fa7-4998-833c-3c70909cd0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:19.801857 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:19.832570 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1444/2901] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:49:19.836696 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc672d25-6870-41f4-99c5-a7b2e4b0a200 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:19.907761 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc672d25-6870-41f4-99c5-a7b2e4b0a200 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:19.912608 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7957a01-3685-42f3-a81b-6f64af1e91a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:19.913191 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7957a01-3685-42f3-a81b-6f64af1e91a0 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:19.913466 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7957a01-3685-42f3-a81b-6f64af1e91a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:19.913732 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7957a01-3685-42f3-a81b-6f64af1e91a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:19.914369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7957a01-3685-42f3-a81b-6f64af1e91a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:19.916835 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7957a01-3685-42f3-a81b-6f64af1e91a0 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:20.039145 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1458/2902] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:49:19 2022] POST /identity/v3/projects => generated 379 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:49:20.043338 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a357df3-2fa7-4998-833c-3c70909cd0b5 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:20.056419 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a357df3-2fa7-4998-833c-3c70909cd0b5 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:20.129082 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f17ab-9a51-4716-8c78-bbd46bad199a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:20.129703 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f17ab-9a51-4716-8c78-bbd46bad199a admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:20.129978 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f17ab-9a51-4716-8c78-bbd46bad199a admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:20.130240 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f17ab-9a51-4716-8c78-bbd46bad199a admin admin] PATH_INFO: `/v3/users` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:20.130918 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f17ab-9a51-4716-8c78-bbd46bad199a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:20.132828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f17ab-9a51-4716-8c78-bbd46bad199a admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:20.218002 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1445/2903] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:49:20 2022] POST /identity/v3/users => generated 296 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:49:20.222622 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7957a01-3685-42f3-a81b-6f64af1e91a0 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:20.233831 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7957a01-3685-42f3-a81b-6f64af1e91a0 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:20.239109 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa0174a-dbc5-48b7-b320-8b41dbfb03b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:20.239723 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0174a-dbc5-48b7-b320-8b41dbfb03b0 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:20.239997 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0174a-dbc5-48b7-b320-8b41dbfb03b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:20.240264 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0174a-dbc5-48b7-b320-8b41dbfb03b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:20.241002 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa0174a-dbc5-48b7-b320-8b41dbfb03b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:20.242628 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa0174a-dbc5-48b7-b320-8b41dbfb03b0 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:20.254559 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1459/2904] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:49:20.259994 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f17ab-9a51-4716-8c78-bbd46bad199a admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:20.271102 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f17ab-9a51-4716-8c78-bbd46bad199a admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:20.276048 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0cf9f-0525-46af-86e2-0d65c6457098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:20.276828 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0cf9f-0525-46af-86e2-0d65c6457098 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:20.277239 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0cf9f-0525-46af-86e2-0d65c6457098 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:20.277610 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0cf9f-0525-46af-86e2-0d65c6457098 admin admin] PATH_INFO: `/v3/projects/44167657c6474b1fa2ccb31dd3b671de/users/054b9af38be54f0dbe534651b7a269ab/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:20.294500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db0cf9f-0525-46af-86e2-0d65c6457098 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44167657c6474b1fa2ccb31dd3b671de, user_id=054b9af38be54f0dbe534651b7a269ab, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:20.297793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db0cf9f-0525-46af-86e2-0d65c6457098 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:20.384835 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1446/2905] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:49:20 2022] PUT /identity/v3/projects/44167657c6474b1fa2ccb31dd3b671de/users/054b9af38be54f0dbe534651b7a269ab/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:49:20.389379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa0174a-dbc5-48b7-b320-8b41dbfb03b0 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:20.403669 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa0174a-dbc5-48b7-b320-8b41dbfb03b0 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:20.488981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705b74e-6b6d-4ae9-9410-f69b6c1147c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:20.489593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705b74e-6b6d-4ae9-9410-f69b6c1147c7 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:20.489872 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705b74e-6b6d-4ae9-9410-f69b6c1147c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:20.490144 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705b74e-6b6d-4ae9-9410-f69b6c1147c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:20.490823 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705b74e-6b6d-4ae9-9410-f69b6c1147c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:20.492457 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705b74e-6b6d-4ae9-9410-f69b6c1147c7 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:20.503591 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1460/2906] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:49:20.508138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0cf9f-0525-46af-86e2-0d65c6457098 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:20.519647 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0cf9f-0525-46af-86e2-0d65c6457098 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:20.524392 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c578eb-a832-49af-9014-c37ca7c0d486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:20.525068 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c578eb-a832-49af-9014-c37ca7c0d486 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:20.525343 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c578eb-a832-49af-9014-c37ca7c0d486 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:20.525616 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c578eb-a832-49af-9014-c37ca7c0d486 admin admin] PATH_INFO: `/v3/projects/44167657c6474b1fa2ccb31dd3b671de/users/054b9af38be54f0dbe534651b7a269ab/roles/13ca33d694c645d6920930d39dc52a0d` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:20.527645 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c578eb-a832-49af-9014-c37ca7c0d486 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44167657c6474b1fa2ccb31dd3b671de, user_id=054b9af38be54f0dbe534651b7a269ab, role_id=13ca33d694c645d6920930d39dc52a0d)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:20.530303 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c578eb-a832-49af-9014-c37ca7c0d486 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:20.547505 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1447/2907] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:49:20 2022] PUT /identity/v3/projects/44167657c6474b1fa2ccb31dd3b671de/users/054b9af38be54f0dbe534651b7a269ab/roles/13ca33d694c645d6920930d39dc52a0d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:49:20.558776 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd616c02-bfc3-4af9-8a82-d496079468e8 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:20.559426 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd616c02-bfc3-4af9-8a82-d496079468e8 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:20.559761 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd616c02-bfc3-4af9-8a82-d496079468e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:20.568902 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:20.592823 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-bd616c02-bfc3-4af9-8a82-d496079468e8 None None] MFA Rules not processed for user `054b9af38be54f0dbe534651b7a269ab`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:20.595451 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-bd616c02-bfc3-4af9-8a82-d496079468e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:20.673598 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1461/2908] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:20 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:49:20.688239 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c578eb-a832-49af-9014-c37ca7c0d486 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:20.690073 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-a3c578eb-a832-49af-9014-c37ca7c0d486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:20.756736 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c578eb-a832-49af-9014-c37ca7c0d486 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:20.761592 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064b93d-de54-459a-884d-5248d13d6984 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:20.762207 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064b93d-de54-459a-884d-5248d13d6984 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:20.762513 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064b93d-de54-459a-884d-5248d13d6984 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:20.762815 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064b93d-de54-459a-884d-5248d13d6984 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:20.774122 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3064b93d-de54-459a-884d-5248d13d6984 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:20.775695 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3064b93d-de54-459a-884d-5248d13d6984 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:20.787469 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1448/2909] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:49:20 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:49:21.463424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd616c02-bfc3-4af9-8a82-d496079468e8 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:21.532567 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd616c02-bfc3-4af9-8a82-d496079468e8 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:21.537270 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f7717-5b19-4511-a69d-6be43b701f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:21.537904 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f7717-5b19-4511-a69d-6be43b701f84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:21.538181 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f7717-5b19-4511-a69d-6be43b701f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:21.538483 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f7717-5b19-4511-a69d-6be43b701f84 admin admin] PATH_INFO: `/v3/users/3f958f45da7a499c9d9b8d6af4a39cc0` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:21.539649 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f7717-5b19-4511-a69d-6be43b701f84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f958f45da7a499c9d9b8d6af4a39cc0)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:21.541624 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f7717-5b19-4511-a69d-6be43b701f84 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:21.727663 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-251f7717-5b19-4511-a69d-6be43b701f84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f958f45da7a499c9d9b8d6af4a39cc0'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:21.741474 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-251f7717-5b19-4511-a69d-6be43b701f84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f958f45da7a499c9d9b8d6af4a39cc0'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:21.749443 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-251f7717-5b19-4511-a69d-6be43b701f84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f958f45da7a499c9d9b8d6af4a39cc0'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:21.824902 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-251f7717-5b19-4511-a69d-6be43b701f84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f958f45da7a499c9d9b8d6af4a39cc0'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:21.825623 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-251f7717-5b19-4511-a69d-6be43b701f84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f958f45da7a499c9d9b8d6af4a39cc0'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:21.833502 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1462/2910] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:49:21 2022] DELETE /identity/v3/users/3f958f45da7a499c9d9b8d6af4a39cc0 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:49:21.838178 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac73fe-0bdb-4088-954d-73959d387a68 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:21.838841 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac73fe-0bdb-4088-954d-73959d387a68 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:21.838841 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac73fe-0bdb-4088-954d-73959d387a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:21.859835 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-bbac73fe-0bdb-4088-954d-73959d387a68 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:21.861310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-bbac73fe-0bdb-4088-954d-73959d387a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:21.907179 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:21.930667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1449/2911] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:49:21.941994 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f7717-5b19-4511-a69d-6be43b701f84 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:21.943505 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-251f7717-5b19-4511-a69d-6be43b701f84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:21.956587 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:22.013367 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f7717-5b19-4511-a69d-6be43b701f84 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:22.018039 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff394b-8a5a-4869-b0ad-e669725ccb40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:22.020779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff394b-8a5a-4869-b0ad-e669725ccb40 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:22.020779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff394b-8a5a-4869-b0ad-e669725ccb40 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:22.020779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff394b-8a5a-4869-b0ad-e669725ccb40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:22.029670 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff394b-8a5a-4869-b0ad-e669725ccb40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:22.031222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff394b-8a5a-4869-b0ad-e669725ccb40 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:22.043271 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1463/2912] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:49:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:49:22.471263 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac73fe-0bdb-4088-954d-73959d387a68 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:22.472907 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-bbac73fe-0bdb-4088-954d-73959d387a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:22.489428 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac73fe-0bdb-4088-954d-73959d387a68 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:22.496735 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a7668-a8e8-40cb-b83a-d4200b7e9000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:22.497638 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a7668-a8e8-40cb-b83a-d4200b7e9000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:22.498134 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a7668-a8e8-40cb-b83a-d4200b7e9000 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:22.498766 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a7668-a8e8-40cb-b83a-d4200b7e9000 admin admin] PATH_INFO: `/v3/projects/18df80ee69704ccda6e8a9c9cde627b0` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:22.503099 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a7668-a8e8-40cb-b83a-d4200b7e9000 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18df80ee69704ccda6e8a9c9cde627b0)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:22.507344 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a7668-a8e8-40cb-b83a-d4200b7e9000 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:22.656505 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-634a7668-a8e8-40cb-b83a-d4200b7e9000 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18df80ee69704ccda6e8a9c9cde627b0', 'request_id': 'req-634a7668-a8e8-40cb-b83a-d4200b7e9000'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:22.666942 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-634a7668-a8e8-40cb-b83a-d4200b7e9000 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18df80ee69704ccda6e8a9c9cde627b0', 'request_id': 'req-634a7668-a8e8-40cb-b83a-d4200b7e9000'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:22.677568 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-634a7668-a8e8-40cb-b83a-d4200b7e9000 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18df80ee69704ccda6e8a9c9cde627b0', 'request_id': 'req-634a7668-a8e8-40cb-b83a-d4200b7e9000'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:22.750627 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-634a7668-a8e8-40cb-b83a-d4200b7e9000 admin admin] The token cache is being invalidate because project 18df80ee69704ccda6e8a9c9cde627b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:49:22.750906 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-634a7668-a8e8-40cb-b83a-d4200b7e9000 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:22.751655 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-634a7668-a8e8-40cb-b83a-d4200b7e9000 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:22.753300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1450/2913] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:49:22 2022] DELETE /identity/v3/projects/18df80ee69704ccda6e8a9c9cde627b0 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:49:22.765139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff394b-8a5a-4869-b0ad-e669725ccb40 service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:22.776790 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff394b-8a5a-4869-b0ad-e669725ccb40 service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:22.848434 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b9fa2-7db4-452d-b053-3deb03a0971a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:22.849152 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b9fa2-7db4-452d-b053-3deb03a0971a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:22.849531 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b9fa2-7db4-452d-b053-3deb03a0971a admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:22.849853 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b9fa2-7db4-452d-b053-3deb03a0971a admin admin] PATH_INFO: `/v3/users/054b9af38be54f0dbe534651b7a269ab` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:22.851099 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b9fa2-7db4-452d-b053-3deb03a0971a admin admin] RBAC: Authorizing `identity:delete_user(user_id=054b9af38be54f0dbe534651b7a269ab)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:22.853644 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b9fa2-7db4-452d-b053-3deb03a0971a admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:23.076555 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-506b9fa2-7db4-452d-b053-3deb03a0971a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '054b9af38be54f0dbe534651b7a269ab'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:23.090970 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-506b9fa2-7db4-452d-b053-3deb03a0971a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '054b9af38be54f0dbe534651b7a269ab'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:23.099977 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-506b9fa2-7db4-452d-b053-3deb03a0971a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '054b9af38be54f0dbe534651b7a269ab'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:23.179569 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-506b9fa2-7db4-452d-b053-3deb03a0971a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '054b9af38be54f0dbe534651b7a269ab'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:23.180231 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-506b9fa2-7db4-452d-b053-3deb03a0971a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '054b9af38be54f0dbe534651b7a269ab'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:23.187553 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1464/2914] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:49:22 2022] DELETE /identity/v3/users/054b9af38be54f0dbe534651b7a269ab => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:49:23.701926 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a7668-a8e8-40cb-b83a-d4200b7e9000 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:23.704155 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-634a7668-a8e8-40cb-b83a-d4200b7e9000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:23.719348 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a7668-a8e8-40cb-b83a-d4200b7e9000 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:23.790629 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457933c8-9f11-4f52-bdfd-b31d2b8579ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:23.791275 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457933c8-9f11-4f52-bdfd-b31d2b8579ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:23.791546 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457933c8-9f11-4f52-bdfd-b31d2b8579ac admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:23.791837 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457933c8-9f11-4f52-bdfd-b31d2b8579ac admin admin] PATH_INFO: `/v3/projects/44167657c6474b1fa2ccb31dd3b671de` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:23.793126 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457933c8-9f11-4f52-bdfd-b31d2b8579ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=44167657c6474b1fa2ccb31dd3b671de)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:23.795194 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457933c8-9f11-4f52-bdfd-b31d2b8579ac admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:23.941840 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-457933c8-9f11-4f52-bdfd-b31d2b8579ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44167657c6474b1fa2ccb31dd3b671de', 'request_id': 'req-457933c8-9f11-4f52-bdfd-b31d2b8579ac'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:23.948663 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-457933c8-9f11-4f52-bdfd-b31d2b8579ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44167657c6474b1fa2ccb31dd3b671de', 'request_id': 'req-457933c8-9f11-4f52-bdfd-b31d2b8579ac'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:23.956291 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-457933c8-9f11-4f52-bdfd-b31d2b8579ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44167657c6474b1fa2ccb31dd3b671de', 'request_id': 'req-457933c8-9f11-4f52-bdfd-b31d2b8579ac'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:23.959996 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:24.061506 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-457933c8-9f11-4f52-bdfd-b31d2b8579ac admin admin] The token cache is being invalidate because project 44167657c6474b1fa2ccb31dd3b671de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:49:24.061806 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-457933c8-9f11-4f52-bdfd-b31d2b8579ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:24.062698 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-457933c8-9f11-4f52-bdfd-b31d2b8579ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:24.064448 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1451/2915] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:49:23 2022] DELETE /identity/v3/projects/44167657c6474b1fa2ccb31dd3b671de => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:49:24.070322 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54660aad-10cb-4eaf-9b0f-d538e40165a8 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:24.070994 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54660aad-10cb-4eaf-9b0f-d538e40165a8 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:24.071135 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54660aad-10cb-4eaf-9b0f-d538e40165a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:24.095252 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-54660aad-10cb-4eaf-9b0f-d538e40165a8 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:24.096505 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-54660aad-10cb-4eaf-9b0f-d538e40165a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:24.134308 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:24.177905 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1465/2916] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:49:24.186517 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a004cb-e50a-45a0-a409-000cff5ee169 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:24.186860 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a004cb-e50a-45a0-a409-000cff5ee169 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:24.187300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a004cb-e50a-45a0-a409-000cff5ee169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:24.210233 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-d7a004cb-e50a-45a0-a409-000cff5ee169 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:24.211675 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-d7a004cb-e50a-45a0-a409-000cff5ee169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:24.221157 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1452/2917] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:49:24.225972 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54660aad-10cb-4eaf-9b0f-d538e40165a8 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:24.236765 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54660aad-10cb-4eaf-9b0f-d538e40165a8 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:24.242093 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22326a-327b-4333-89d7-768bd496b099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:24.244787 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22326a-327b-4333-89d7-768bd496b099 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:24.244787 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22326a-327b-4333-89d7-768bd496b099 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:24.244787 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22326a-327b-4333-89d7-768bd496b099 admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:24.244787 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a22326a-327b-4333-89d7-768bd496b099 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:49:24.244787 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a22326a-327b-4333-89d7-768bd496b099 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:24.246209 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a22326a-327b-4333-89d7-768bd496b099 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:24.260119 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1466/2918] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:49:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:49:24.265375 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a004cb-e50a-45a0-a409-000cff5ee169 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:24.276439 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a004cb-e50a-45a0-a409-000cff5ee169 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:24.281637 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aaba80-131b-48e1-b551-3990bed9584d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:24.282346 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aaba80-131b-48e1-b551-3990bed9584d admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:24.282667 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aaba80-131b-48e1-b551-3990bed9584d admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:24.282936 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aaba80-131b-48e1-b551-3990bed9584d admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:24.283592 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aaba80-131b-48e1-b551-3990bed9584d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:24.285641 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aaba80-131b-48e1-b551-3990bed9584d admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:24.416934 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1453/2919] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:49:24 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:49:24.421192 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22326a-327b-4333-89d7-768bd496b099 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:24.436148 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22326a-327b-4333-89d7-768bd496b099 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:24.504473 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c322410b-7865-4167-b69d-67bfcb5d5e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:24.505004 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322410b-7865-4167-b69d-67bfcb5d5e07 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:24.505280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322410b-7865-4167-b69d-67bfcb5d5e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:24.505552 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322410b-7865-4167-b69d-67bfcb5d5e07 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:24.506969 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c322410b-7865-4167-b69d-67bfcb5d5e07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:24.508976 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c322410b-7865-4167-b69d-67bfcb5d5e07 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:24.617994 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1467/2920] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:49:24 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:49:24.623225 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aaba80-131b-48e1-b551-3990bed9584d admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:24.643478 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aaba80-131b-48e1-b551-3990bed9584d admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:24.649829 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0631e5-8ee9-4c61-9d09-15a75550716a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:24.650518 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0631e5-8ee9-4c61-9d09-15a75550716a admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:24.650793 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0631e5-8ee9-4c61-9d09-15a75550716a admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:24.651060 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0631e5-8ee9-4c61-9d09-15a75550716a admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:24.651807 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0631e5-8ee9-4c61-9d09-15a75550716a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:24.653460 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0631e5-8ee9-4c61-9d09-15a75550716a admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:24.665627 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1454/2921] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:49:24.670347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c322410b-7865-4167-b69d-67bfcb5d5e07 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:24.682169 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c322410b-7865-4167-b69d-67bfcb5d5e07 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:24.687846 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa93a98-fcb9-4bf5-ba29-df06c6ba4929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:24.688830 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa93a98-fcb9-4bf5-ba29-df06c6ba4929 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:24.689116 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa93a98-fcb9-4bf5-ba29-df06c6ba4929 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:24.689391 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa93a98-fcb9-4bf5-ba29-df06c6ba4929 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:24.690091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa93a98-fcb9-4bf5-ba29-df06c6ba4929 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:24.691742 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa93a98-fcb9-4bf5-ba29-df06c6ba4929 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:24.703394 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1468/2922] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:49:24.707748 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0631e5-8ee9-4c61-9d09-15a75550716a admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:24.718086 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0631e5-8ee9-4c61-9d09-15a75550716a admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:24.728175 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8238ff-83eb-4ed9-b4f8-6dde3eba3ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:24.729057 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8238ff-83eb-4ed9-b4f8-6dde3eba3ba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:24.729431 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8238ff-83eb-4ed9-b4f8-6dde3eba3ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:24.729907 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8238ff-83eb-4ed9-b4f8-6dde3eba3ba5 admin admin] PATH_INFO: `/v3/projects/7dc615c0f88d4c19828bce771566c31c/users/ca120e543c9148b8957a65896eb44673/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:24.751538 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8238ff-83eb-4ed9-b4f8-6dde3eba3ba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dc615c0f88d4c19828bce771566c31c, user_id=ca120e543c9148b8957a65896eb44673, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:24.754220 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8238ff-83eb-4ed9-b4f8-6dde3eba3ba5 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:24.830031 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1455/2923] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:49:24 2022] PUT /identity/v3/projects/7dc615c0f88d4c19828bce771566c31c/users/ca120e543c9148b8957a65896eb44673/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:49:24.840490 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940a6c8-de0c-412b-bdd8-ee41f218c28d None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:24.840779 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940a6c8-de0c-412b-bdd8-ee41f218c28d None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:24.841055 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940a6c8-de0c-412b-bdd8-ee41f218c28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:24.872012 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-6940a6c8-de0c-412b-bdd8-ee41f218c28d None None] MFA Rules not processed for user `ca120e543c9148b8957a65896eb44673`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:24.873236 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-6940a6c8-de0c-412b-bdd8-ee41f218c28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:24.943551 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1469/2924] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:49:24.956518 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8238ff-83eb-4ed9-b4f8-6dde3eba3ba5 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:24.958559 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-7c8238ff-83eb-4ed9-b4f8-6dde3eba3ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:25.037004 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8238ff-83eb-4ed9-b4f8-6dde3eba3ba5 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:25.040724 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c44f3-f940-4844-aab9-4491d22be887 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:25.041386 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c44f3-f940-4844-aab9-4491d22be887 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:25.041661 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c44f3-f940-4844-aab9-4491d22be887 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:25.041930 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c44f3-f940-4844-aab9-4491d22be887 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:25.051511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c44f3-f940-4844-aab9-4491d22be887 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:25.052971 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c44f3-f940-4844-aab9-4491d22be887 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:25.064268 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1456/2925] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:49:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:49:25.086160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6940a6c8-de0c-412b-bdd8-ee41f218c28d None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:25.127762 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:25.166365 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6940a6c8-de0c-412b-bdd8-ee41f218c28d None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:25.171524 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4ae47-7e21-493a-a0a4-61e130018da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:25.172236 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4ae47-7e21-493a-a0a4-61e130018da4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:25.172516 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4ae47-7e21-493a-a0a4-61e130018da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:25.172789 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4ae47-7e21-493a-a0a4-61e130018da4 admin admin] PATH_INFO: `/v3/users/ca120e543c9148b8957a65896eb44673` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:25.173991 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4ae47-7e21-493a-a0a4-61e130018da4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca120e543c9148b8957a65896eb44673)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:25.176080 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4ae47-7e21-493a-a0a4-61e130018da4 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:25.366828 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-d3e4ae47-7e21-493a-a0a4-61e130018da4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca120e543c9148b8957a65896eb44673'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:25.380666 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-d3e4ae47-7e21-493a-a0a4-61e130018da4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca120e543c9148b8957a65896eb44673'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:25.388508 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-d3e4ae47-7e21-493a-a0a4-61e130018da4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca120e543c9148b8957a65896eb44673'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:25.470900 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-d3e4ae47-7e21-493a-a0a4-61e130018da4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca120e543c9148b8957a65896eb44673'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:25.471648 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-d3e4ae47-7e21-493a-a0a4-61e130018da4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca120e543c9148b8957a65896eb44673'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:25.479442 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1470/2926] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:49:25 2022] DELETE /identity/v3/users/ca120e543c9148b8957a65896eb44673 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:49:25.484855 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69502480-526a-4da5-afa5-2d4fc413ffa8 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:25.485138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69502480-526a-4da5-afa5-2d4fc413ffa8 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:25.485411 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69502480-526a-4da5-afa5-2d4fc413ffa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:25.506635 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-69502480-526a-4da5-afa5-2d4fc413ffa8 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:25.507878 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-69502480-526a-4da5-afa5-2d4fc413ffa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:25.572328 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1457/2927] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:49:25.582455 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4ae47-7e21-493a-a0a4-61e130018da4 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:25.583909 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-d3e4ae47-7e21-493a-a0a4-61e130018da4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:25.645408 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4ae47-7e21-493a-a0a4-61e130018da4 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:25.650081 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4634ad-727a-4624-8760-1edea7fd7ac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:25.650752 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4634ad-727a-4624-8760-1edea7fd7ac1 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:25.651028 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4634ad-727a-4624-8760-1edea7fd7ac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:25.651301 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4634ad-727a-4624-8760-1edea7fd7ac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:25.661744 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4634ad-727a-4624-8760-1edea7fd7ac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:25.663223 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4634ad-727a-4624-8760-1edea7fd7ac1 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:25.674098 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1471/2928] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:49:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:49:26.154073 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69502480-526a-4da5-afa5-2d4fc413ffa8 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:26.156005 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-69502480-526a-4da5-afa5-2d4fc413ffa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:26.172212 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69502480-526a-4da5-afa5-2d4fc413ffa8 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:26.177534 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a637c-4347-43c9-b8d2-5763000ea0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:26.178148 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a637c-4347-43c9-b8d2-5763000ea0a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:26.178450 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a637c-4347-43c9-b8d2-5763000ea0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:26.178729 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a637c-4347-43c9-b8d2-5763000ea0a0 admin admin] PATH_INFO: `/v3/projects/7dc615c0f88d4c19828bce771566c31c` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:26.179981 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a637c-4347-43c9-b8d2-5763000ea0a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dc615c0f88d4c19828bce771566c31c)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:26.183309 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a637c-4347-43c9-b8d2-5763000ea0a0 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:26.287719 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:26.301679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-1d3a637c-4347-43c9-b8d2-5763000ea0a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dc615c0f88d4c19828bce771566c31c', 'request_id': 'req-1d3a637c-4347-43c9-b8d2-5763000ea0a0'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:26.308019 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-1d3a637c-4347-43c9-b8d2-5763000ea0a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dc615c0f88d4c19828bce771566c31c', 'request_id': 'req-1d3a637c-4347-43c9-b8d2-5763000ea0a0'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:26.314843 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-1d3a637c-4347-43c9-b8d2-5763000ea0a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dc615c0f88d4c19828bce771566c31c', 'request_id': 'req-1d3a637c-4347-43c9-b8d2-5763000ea0a0'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:26.391276 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-1d3a637c-4347-43c9-b8d2-5763000ea0a0 admin admin] The token cache is being invalidate because project 7dc615c0f88d4c19828bce771566c31c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:49:26.391695 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-1d3a637c-4347-43c9-b8d2-5763000ea0a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:26.392610 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-1d3a637c-4347-43c9-b8d2-5763000ea0a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:26.394336 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1458/2929] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:49:26 2022] DELETE /identity/v3/projects/7dc615c0f88d4c19828bce771566c31c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:49:26.400053 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abfba4-16d4-4363-a8ee-6ce704e7e5f3 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:26.400053 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abfba4-16d4-4363-a8ee-6ce704e7e5f3 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:26.400807 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abfba4-16d4-4363-a8ee-6ce704e7e5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:26.422168 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-a2abfba4-16d4-4363-a8ee-6ce704e7e5f3 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:26.423685 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a2abfba4-16d4-4363-a8ee-6ce704e7e5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:26.479696 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:26.496430 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1472/2930] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:49:26.505998 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bedc65-33ca-4df3-ab5c-b15a8fd4a496 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:26.506279 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bedc65-33ca-4df3-ab5c-b15a8fd4a496 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:26.506567 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bedc65-33ca-4df3-ab5c-b15a8fd4a496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:26.528316 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-69bedc65-33ca-4df3-ab5c-b15a8fd4a496 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:26.533413 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-69bedc65-33ca-4df3-ab5c-b15a8fd4a496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:26.538987 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1459/2931] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:49:26.544160 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2abfba4-16d4-4363-a8ee-6ce704e7e5f3 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:26.555068 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2abfba4-16d4-4363-a8ee-6ce704e7e5f3 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:26.560868 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1492110a-9fa5-40c5-9b38-adb80b3e9685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:26.561480 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492110a-9fa5-40c5-9b38-adb80b3e9685 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:26.561753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492110a-9fa5-40c5-9b38-adb80b3e9685 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:26.562094 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492110a-9fa5-40c5-9b38-adb80b3e9685 admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:26.562829 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1492110a-9fa5-40c5-9b38-adb80b3e9685 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:49:26.563594 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1492110a-9fa5-40c5-9b38-adb80b3e9685 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:26.567936 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1492110a-9fa5-40c5-9b38-adb80b3e9685 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:26.581633 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1473/2932] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:49:26.585844 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bedc65-33ca-4df3-ab5c-b15a8fd4a496 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:26.597107 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bedc65-33ca-4df3-ab5c-b15a8fd4a496 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:26.602263 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e78cec-8d9f-47fd-a8b6-0d875d5709a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:26.603371 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e78cec-8d9f-47fd-a8b6-0d875d5709a8 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:26.603650 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e78cec-8d9f-47fd-a8b6-0d875d5709a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:26.603920 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e78cec-8d9f-47fd-a8b6-0d875d5709a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:26.604575 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e78cec-8d9f-47fd-a8b6-0d875d5709a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:26.606523 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e78cec-8d9f-47fd-a8b6-0d875d5709a8 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:26.739941 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1460/2933] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:49:26 2022] POST /identity/v3/projects => generated 361 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:49:26.744621 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1492110a-9fa5-40c5-9b38-adb80b3e9685 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:26.756208 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1492110a-9fa5-40c5-9b38-adb80b3e9685 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:26.822424 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48da8401-30f5-41e8-a45e-75e45bb8c676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:26.823058 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da8401-30f5-41e8-a45e-75e45bb8c676 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:26.823330 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da8401-30f5-41e8-a45e-75e45bb8c676 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:26.823603 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da8401-30f5-41e8-a45e-75e45bb8c676 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:26.824257 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48da8401-30f5-41e8-a45e-75e45bb8c676 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:26.826220 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48da8401-30f5-41e8-a45e-75e45bb8c676 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:26.944210 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1474/2934] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:49:26 2022] POST /identity/v3/users => generated 281 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:49:26.949414 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e78cec-8d9f-47fd-a8b6-0d875d5709a8 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:26.961121 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e78cec-8d9f-47fd-a8b6-0d875d5709a8 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:26.966035 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b1bcdc-5ea2-4402-a993-4ee294f02288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:26.967305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1bcdc-5ea2-4402-a993-4ee294f02288 admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:26.967305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1bcdc-5ea2-4402-a993-4ee294f02288 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:26.967305 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1bcdc-5ea2-4402-a993-4ee294f02288 admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:26.967920 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b1bcdc-5ea2-4402-a993-4ee294f02288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:26.969545 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b1bcdc-5ea2-4402-a993-4ee294f02288 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:26.981055 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1461/2935] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:49:26.988026 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48da8401-30f5-41e8-a45e-75e45bb8c676 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:27.003996 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48da8401-30f5-41e8-a45e-75e45bb8c676 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:27.009134 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b58093-6342-4be4-882d-a7cce012ce1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:27.009773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b58093-6342-4be4-882d-a7cce012ce1f admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:27.010084 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b58093-6342-4be4-882d-a7cce012ce1f admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:27.010360 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b58093-6342-4be4-882d-a7cce012ce1f admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:27.011187 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b58093-6342-4be4-882d-a7cce012ce1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:27.013033 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b58093-6342-4be4-882d-a7cce012ce1f admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:27.025465 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1475/2936] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:49:27.030519 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b1bcdc-5ea2-4402-a993-4ee294f02288 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:27.041962 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b1bcdc-5ea2-4402-a993-4ee294f02288 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:27.047891 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7956d27c-f7fa-4f21-b725-be7131b6797e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:27.048588 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7956d27c-f7fa-4f21-b725-be7131b6797e admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:27.048868 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7956d27c-f7fa-4f21-b725-be7131b6797e admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:27.049146 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7956d27c-f7fa-4f21-b725-be7131b6797e admin admin] PATH_INFO: `/v3/projects/29a8761a5773461f85e735682b174210/users/8e33663a97c64dfdb4a169dec4064ad7/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:27.069768 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7956d27c-f7fa-4f21-b725-be7131b6797e admin admin] RBAC: Authorizing `identity:create_grant(project_id=29a8761a5773461f85e735682b174210, user_id=8e33663a97c64dfdb4a169dec4064ad7, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:27.072590 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7956d27c-f7fa-4f21-b725-be7131b6797e admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:27.191038 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1462/2937] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:49:27 2022] PUT /identity/v3/projects/29a8761a5773461f85e735682b174210/users/8e33663a97c64dfdb4a169dec4064ad7/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:49:27.202308 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d515b0f5-669b-47eb-aeca-077e3337c530 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:27.202793 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d515b0f5-669b-47eb-aeca-077e3337c530 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:27.203194 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d515b0f5-669b-47eb-aeca-077e3337c530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:27.240353 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-d515b0f5-669b-47eb-aeca-077e3337c530 None None] MFA Rules not processed for user `8e33663a97c64dfdb4a169dec4064ad7`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:27.242146 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-d515b0f5-669b-47eb-aeca-077e3337c530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:27.318635 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1476/2938] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:49:27.358538 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7956d27c-f7fa-4f21-b725-be7131b6797e admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:27.361415 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-7956d27c-f7fa-4f21-b725-be7131b6797e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:27.428434 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7956d27c-f7fa-4f21-b725-be7131b6797e admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:27.432328 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d76c2f-076f-4c79-9538-2b65f03d01d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:27.432947 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d76c2f-076f-4c79-9538-2b65f03d01d6 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:27.433227 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d76c2f-076f-4c79-9538-2b65f03d01d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:27.433503 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d76c2f-076f-4c79-9538-2b65f03d01d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:27.444606 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d76c2f-076f-4c79-9538-2b65f03d01d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:27.446278 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d76c2f-076f-4c79-9538-2b65f03d01d6 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:27.457374 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1463/2939] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:49:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:49:27.483659 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d515b0f5-669b-47eb-aeca-077e3337c530 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:27.535020 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:27.554612 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d515b0f5-669b-47eb-aeca-077e3337c530 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:27.559680 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b52d6-a121-4bef-acd3-f99e28e02973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:27.560375 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b52d6-a121-4bef-acd3-f99e28e02973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:27.560641 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b52d6-a121-4bef-acd3-f99e28e02973 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:27.560909 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b52d6-a121-4bef-acd3-f99e28e02973 admin admin] PATH_INFO: `/v3/users/8e33663a97c64dfdb4a169dec4064ad7` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:27.562139 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b52d6-a121-4bef-acd3-f99e28e02973 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e33663a97c64dfdb4a169dec4064ad7)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:27.564113 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b52d6-a121-4bef-acd3-f99e28e02973 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:27.784484 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-581b52d6-a121-4bef-acd3-f99e28e02973 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e33663a97c64dfdb4a169dec4064ad7'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:27.802797 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-581b52d6-a121-4bef-acd3-f99e28e02973 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e33663a97c64dfdb4a169dec4064ad7'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:27.813728 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-581b52d6-a121-4bef-acd3-f99e28e02973 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e33663a97c64dfdb4a169dec4064ad7'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:27.916325 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-581b52d6-a121-4bef-acd3-f99e28e02973 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e33663a97c64dfdb4a169dec4064ad7'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:27.916954 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-581b52d6-a121-4bef-acd3-f99e28e02973 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e33663a97c64dfdb4a169dec4064ad7'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:27.927851 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1477/2940] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:49:27 2022] DELETE /identity/v3/users/8e33663a97c64dfdb4a169dec4064ad7 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:49:27.933814 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace709c4-3d8c-49d1-aed2-1bf18a2f8954 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:27.934450 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace709c4-3d8c-49d1-aed2-1bf18a2f8954 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:27.934964 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace709c4-3d8c-49d1-aed2-1bf18a2f8954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:27.956995 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-ace709c4-3d8c-49d1-aed2-1bf18a2f8954 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:27.960725 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-ace709c4-3d8c-49d1-aed2-1bf18a2f8954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:28.024160 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:28.039716 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1464/2941] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:49:28.050937 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b52d6-a121-4bef-acd3-f99e28e02973 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:28.052736 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-581b52d6-a121-4bef-acd3-f99e28e02973 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:28.132626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b52d6-a121-4bef-acd3-f99e28e02973 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:28.136627 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a670988-98b4-4cec-9a67-030789a8efa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:28.137257 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a670988-98b4-4cec-9a67-030789a8efa0 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:28.137636 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a670988-98b4-4cec-9a67-030789a8efa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:28.137913 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a670988-98b4-4cec-9a67-030789a8efa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:28.148493 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a670988-98b4-4cec-9a67-030789a8efa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:28.149963 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a670988-98b4-4cec-9a67-030789a8efa0 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:28.161953 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1478/2942] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:49:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:49:28.684392 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace709c4-3d8c-49d1-aed2-1bf18a2f8954 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:28.686360 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-ace709c4-3d8c-49d1-aed2-1bf18a2f8954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:28.700315 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace709c4-3d8c-49d1-aed2-1bf18a2f8954 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:28.707815 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64767d1d-7d87-4e6d-8feb-83a4ae6834bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:28.710083 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64767d1d-7d87-4e6d-8feb-83a4ae6834bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:28.710083 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64767d1d-7d87-4e6d-8feb-83a4ae6834bc admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:28.710083 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64767d1d-7d87-4e6d-8feb-83a4ae6834bc admin admin] PATH_INFO: `/v3/projects/29a8761a5773461f85e735682b174210` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:28.712087 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64767d1d-7d87-4e6d-8feb-83a4ae6834bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=29a8761a5773461f85e735682b174210)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:28.716047 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64767d1d-7d87-4e6d-8feb-83a4ae6834bc admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:28.859528 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-64767d1d-7d87-4e6d-8feb-83a4ae6834bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29a8761a5773461f85e735682b174210', 'request_id': 'req-64767d1d-7d87-4e6d-8feb-83a4ae6834bc'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:28.873801 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-64767d1d-7d87-4e6d-8feb-83a4ae6834bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29a8761a5773461f85e735682b174210', 'request_id': 'req-64767d1d-7d87-4e6d-8feb-83a4ae6834bc'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:28.889565 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-64767d1d-7d87-4e6d-8feb-83a4ae6834bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29a8761a5773461f85e735682b174210', 'request_id': 'req-64767d1d-7d87-4e6d-8feb-83a4ae6834bc'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:28.982118 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-64767d1d-7d87-4e6d-8feb-83a4ae6834bc admin admin] The token cache is being invalidate because project 29a8761a5773461f85e735682b174210 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:49:28.982442 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-64767d1d-7d87-4e6d-8feb-83a4ae6834bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:28.983171 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-64767d1d-7d87-4e6d-8feb-83a4ae6834bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:49:28.985291 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1465/2943] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:49:28 2022] DELETE /identity/v3/projects/29a8761a5773461f85e735682b174210 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:49:28.990786 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b0b25-222f-4bfc-8550-718779f696b1 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:28.991581 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b0b25-222f-4bfc-8550-718779f696b1 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:28.991747 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b0b25-222f-4bfc-8550-718779f696b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:29.022289 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-209b0b25-222f-4bfc-8550-718779f696b1 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:29.023795 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-209b0b25-222f-4bfc-8550-718779f696b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:29.116472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1479/2944] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:49:29.125267 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcbe55-9548-4519-b9a5-ed5b61dfbb90 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:29.125570 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcbe55-9548-4519-b9a5-ed5b61dfbb90 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:29.125858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcbe55-9548-4519-b9a5-ed5b61dfbb90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:29.149370 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-7adcbe55-9548-4519-b9a5-ed5b61dfbb90 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:29.150687 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-7adcbe55-9548-4519-b9a5-ed5b61dfbb90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:29.160078 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1466/2945] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:49:29.170316 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b0b25-222f-4bfc-8550-718779f696b1 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:29.181590 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b0b25-222f-4bfc-8550-718779f696b1 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:29.187048 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6212552-d568-4653-9d95-9875858f143a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:29.188092 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6212552-d568-4653-9d95-9875858f143a admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:29.195988 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6212552-d568-4653-9d95-9875858f143a admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:29.195988 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6212552-d568-4653-9d95-9875858f143a admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:29.195988 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6212552-d568-4653-9d95-9875858f143a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:49:29.195988 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6212552-d568-4653-9d95-9875858f143a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:29.195988 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6212552-d568-4653-9d95-9875858f143a admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:29.195988 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:29.208307 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1480/2946] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:49:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:49:29.213029 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcbe55-9548-4519-b9a5-ed5b61dfbb90 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:29.224048 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcbe55-9548-4519-b9a5-ed5b61dfbb90 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:29.234727 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08704be-4ab7-411d-89bb-6f6b7ed2eeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:29.234727 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08704be-4ab7-411d-89bb-6f6b7ed2eeb6 admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:29.234727 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08704be-4ab7-411d-89bb-6f6b7ed2eeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:29.234727 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08704be-4ab7-411d-89bb-6f6b7ed2eeb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:29.235465 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08704be-4ab7-411d-89bb-6f6b7ed2eeb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:29.235465 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08704be-4ab7-411d-89bb-6f6b7ed2eeb6 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:29.358496 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1467/2947] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:49:29 2022] POST /identity/v3/projects => generated 377 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:49:29.363818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6212552-d568-4653-9d95-9875858f143a admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:29.375995 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6212552-d568-4653-9d95-9875858f143a admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:29.442161 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e603b-8228-4fde-9f2b-44638cd9ec97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:29.442951 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e603b-8228-4fde-9f2b-44638cd9ec97 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:29.443274 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e603b-8228-4fde-9f2b-44638cd9ec97 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:29.443590 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e603b-8228-4fde-9f2b-44638cd9ec97 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:29.444304 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8e603b-8228-4fde-9f2b-44638cd9ec97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:29.446332 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8e603b-8228-4fde-9f2b-44638cd9ec97 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:29.585743 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1481/2948] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:49:29 2022] POST /identity/v3/users => generated 289 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:49:29.590295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08704be-4ab7-411d-89bb-6f6b7ed2eeb6 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:29.602628 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08704be-4ab7-411d-89bb-6f6b7ed2eeb6 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:29.608100 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a40f8-231e-4dfc-b4af-f0864d8bd32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:29.609101 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a40f8-231e-4dfc-b4af-f0864d8bd32f admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:29.609367 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a40f8-231e-4dfc-b4af-f0864d8bd32f admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:29.610043 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a40f8-231e-4dfc-b4af-f0864d8bd32f admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:29.610740 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a40f8-231e-4dfc-b4af-f0864d8bd32f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:29.612467 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a40f8-231e-4dfc-b4af-f0864d8bd32f admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:29.623379 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1468/2949] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:49:29.628355 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e603b-8228-4fde-9f2b-44638cd9ec97 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:29.639063 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e603b-8228-4fde-9f2b-44638cd9ec97 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:29.644792 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403074f8-9b5c-4970-a337-852e8e8df546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:29.645416 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403074f8-9b5c-4970-a337-852e8e8df546 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:29.645720 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403074f8-9b5c-4970-a337-852e8e8df546 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:29.645995 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403074f8-9b5c-4970-a337-852e8e8df546 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:29.646676 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403074f8-9b5c-4970-a337-852e8e8df546 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:29.648773 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403074f8-9b5c-4970-a337-852e8e8df546 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:29.661047 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1482/2950] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:49:29.665529 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a40f8-231e-4dfc-b4af-f0864d8bd32f admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:29.676910 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a40f8-231e-4dfc-b4af-f0864d8bd32f admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:29.681551 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1428a6b-ad6f-4257-8534-43518ac35834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:29.682221 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1428a6b-ad6f-4257-8534-43518ac35834 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:29.683591 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1428a6b-ad6f-4257-8534-43518ac35834 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:29.683591 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1428a6b-ad6f-4257-8534-43518ac35834 admin admin] PATH_INFO: `/v3/projects/cf31c8497b6e4a5e865fc3df6d5e47e1/users/7b458a47c2da4e5086f96c9cbe9f7e81/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:29.700138 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1428a6b-ad6f-4257-8534-43518ac35834 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf31c8497b6e4a5e865fc3df6d5e47e1, user_id=7b458a47c2da4e5086f96c9cbe9f7e81, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:29.702931 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1428a6b-ad6f-4257-8534-43518ac35834 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:29.790097 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1469/2951] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:49:29 2022] PUT /identity/v3/projects/cf31c8497b6e4a5e865fc3df6d5e47e1/users/7b458a47c2da4e5086f96c9cbe9f7e81/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:49:29.802593 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb36d6d-643b-49fa-b664-332f03e56550 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:29.802935 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb36d6d-643b-49fa-b664-332f03e56550 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:29.803277 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb36d6d-643b-49fa-b664-332f03e56550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:29.835615 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-2fb36d6d-643b-49fa-b664-332f03e56550 None None] MFA Rules not processed for user `7b458a47c2da4e5086f96c9cbe9f7e81`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:49:29.836918 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-2fb36d6d-643b-49fa-b664-332f03e56550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:29.917530 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1483/2952] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:49:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:49:29.930055 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1428a6b-ad6f-4257-8534-43518ac35834 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:49:29.932039 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-e1428a6b-ad6f-4257-8534-43518ac35834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:49:30.014318 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1428a6b-ad6f-4257-8534-43518ac35834 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:49:30.018265 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae034aa-ec26-4abe-9efb-a2612218f61e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:49:30.018985 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae034aa-ec26-4abe-9efb-a2612218f61e service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:49:30.019301 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae034aa-ec26-4abe-9efb-a2612218f61e service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:49:30.019609 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae034aa-ec26-4abe-9efb-a2612218f61e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:49:30.029488 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae034aa-ec26-4abe-9efb-a2612218f61e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:49:30.031083 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae034aa-ec26-4abe-9efb-a2612218f61e service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:49:30.041921 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1470/2953] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:49:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:49:39.912875 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=69,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:49:40.036777 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:50:03.332617 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb36d6d-643b-49fa-b664-332f03e56550 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:50:03.437981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb36d6d-643b-49fa-b664-332f03e56550 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:50:03.442711 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd402ea-2ae4-4e5c-9b6a-ff49103296d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:50:03.443357 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd402ea-2ae4-4e5c-9b6a-ff49103296d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:50:03.443633 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd402ea-2ae4-4e5c-9b6a-ff49103296d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:50:03.443903 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd402ea-2ae4-4e5c-9b6a-ff49103296d0 admin admin] PATH_INFO: `/v3/users/7b458a47c2da4e5086f96c9cbe9f7e81` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:50:03.445078 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd402ea-2ae4-4e5c-9b6a-ff49103296d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b458a47c2da4e5086f96c9cbe9f7e81)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:50:03.447097 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd402ea-2ae4-4e5c-9b6a-ff49103296d0 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:50:03.620361 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ffd402ea-2ae4-4e5c-9b6a-ff49103296d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b458a47c2da4e5086f96c9cbe9f7e81'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:50:03.635302 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ffd402ea-2ae4-4e5c-9b6a-ff49103296d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b458a47c2da4e5086f96c9cbe9f7e81'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:50:03.643347 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ffd402ea-2ae4-4e5c-9b6a-ff49103296d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b458a47c2da4e5086f96c9cbe9f7e81'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:50:03.704241 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ffd402ea-2ae4-4e5c-9b6a-ff49103296d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b458a47c2da4e5086f96c9cbe9f7e81'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:50:03.705052 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-ffd402ea-2ae4-4e5c-9b6a-ff49103296d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b458a47c2da4e5086f96c9cbe9f7e81'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:50:03.712622 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1484/2954] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:50:03 2022] DELETE /identity/v3/users/7b458a47c2da4e5086f96c9cbe9f7e81 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:50:03.717631 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81c0df-4cdf-4628-9c08-693f12e2b9cc None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:50:03.718234 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81c0df-4cdf-4628-9c08-693f12e2b9cc None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:50:03.718234 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81c0df-4cdf-4628-9c08-693f12e2b9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:50:03.766768 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-7e81c0df-4cdf-4628-9c08-693f12e2b9cc None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:50:03.767922 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-7e81c0df-4cdf-4628-9c08-693f12e2b9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:50:03.833615 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1471/2955] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:50:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:50:03.844091 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd402ea-2ae4-4e5c-9b6a-ff49103296d0 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:50:03.845525 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-ffd402ea-2ae4-4e5c-9b6a-ff49103296d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:50:03.930434 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd402ea-2ae4-4e5c-9b6a-ff49103296d0 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:50:03.934606 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b2bca-acd9-482e-8277-c62244c23bae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:50:03.935238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b2bca-acd9-482e-8277-c62244c23bae service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:50:03.936123 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b2bca-acd9-482e-8277-c62244c23bae service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:50:03.936488 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b2bca-acd9-482e-8277-c62244c23bae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:50:03.944530 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:50:03.948025 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b2bca-acd9-482e-8277-c62244c23bae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:50:03.949280 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b2bca-acd9-482e-8277-c62244c23bae service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:50:03.960394 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1485/2956] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:50:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:50:04.462679 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e81c0df-4cdf-4628-9c08-693f12e2b9cc None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:50:04.483310 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-7e81c0df-4cdf-4628-9c08-693f12e2b9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:50:04.488659 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e81c0df-4cdf-4628-9c08-693f12e2b9cc None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:50:04.497028 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1ab3c-4895-4ea3-bd98-bc5211e569bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:50:04.498722 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1ab3c-4895-4ea3-bd98-bc5211e569bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:50:04.499113 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1ab3c-4895-4ea3-bd98-bc5211e569bf admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:50:04.499507 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1ab3c-4895-4ea3-bd98-bc5211e569bf admin admin] PATH_INFO: `/v3/projects/cf31c8497b6e4a5e865fc3df6d5e47e1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:50:04.501270 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1ab3c-4895-4ea3-bd98-bc5211e569bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf31c8497b6e4a5e865fc3df6d5e47e1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:50:04.503670 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1ab3c-4895-4ea3-bd98-bc5211e569bf admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:50:04.642206 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-39d1ab3c-4895-4ea3-bd98-bc5211e569bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf31c8497b6e4a5e865fc3df6d5e47e1', 'request_id': 'req-39d1ab3c-4895-4ea3-bd98-bc5211e569bf'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:50:04.648792 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-39d1ab3c-4895-4ea3-bd98-bc5211e569bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf31c8497b6e4a5e865fc3df6d5e47e1', 'request_id': 'req-39d1ab3c-4895-4ea3-bd98-bc5211e569bf'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:50:04.655084 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-39d1ab3c-4895-4ea3-bd98-bc5211e569bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf31c8497b6e4a5e865fc3df6d5e47e1', 'request_id': 'req-39d1ab3c-4895-4ea3-bd98-bc5211e569bf'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:50:04.739190 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-39d1ab3c-4895-4ea3-bd98-bc5211e569bf admin admin] The token cache is being invalidate because project cf31c8497b6e4a5e865fc3df6d5e47e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:50:04.739854 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-39d1ab3c-4895-4ea3-bd98-bc5211e569bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:50:04.740104 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-39d1ab3c-4895-4ea3-bd98-bc5211e569bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:50:04.741729 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1472/2957] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:50:04 2022] DELETE /identity/v3/projects/cf31c8497b6e4a5e865fc3df6d5e47e1 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:50:04.747223 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2deda7b-6808-41bc-8ea2-4400493070a8 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:50:04.747827 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2deda7b-6808-41bc-8ea2-4400493070a8 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:50:04.747827 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2deda7b-6808-41bc-8ea2-4400493070a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:50:04.804845 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-f2deda7b-6808-41bc-8ea2-4400493070a8 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:50:04.806143 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-f2deda7b-6808-41bc-8ea2-4400493070a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:50:04.882960 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1486/2958] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:50:04.891573 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c5d6a-5a6c-4ab5-a9b0-df184e552ce3 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:50:04.891869 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c5d6a-5a6c-4ab5-a9b0-df184e552ce3 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:50:04.892124 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c5d6a-5a6c-4ab5-a9b0-df184e552ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:50:04.917081 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-564c5d6a-5a6c-4ab5-a9b0-df184e552ce3 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:50:04.918343 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-564c5d6a-5a6c-4ab5-a9b0-df184e552ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:50:04.927939 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1473/2959] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:50:04.932646 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2deda7b-6808-41bc-8ea2-4400493070a8 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:50:04.947029 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2deda7b-6808-41bc-8ea2-4400493070a8 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:50:04.955412 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb0645a-ef02-4cce-b0e8-14b5b3efc33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:50:04.956472 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0645a-ef02-4cce-b0e8-14b5b3efc33a admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:50:04.956967 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0645a-ef02-4cce-b0e8-14b5b3efc33a admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:50:04.957462 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0645a-ef02-4cce-b0e8-14b5b3efc33a admin admin] PATH_INFO: `/v3/domains` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:50:04.958713 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb0645a-ef02-4cce-b0e8-14b5b3efc33a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:50:04.959286 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb0645a-ef02-4cce-b0e8-14b5b3efc33a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:50:04.962415 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb0645a-ef02-4cce-b0e8-14b5b3efc33a admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:50:04.977753 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1487/2960] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 31 11:50:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:50:04.983595 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c5d6a-5a6c-4ab5-a9b0-df184e552ce3 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:50:04.993531 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c5d6a-5a6c-4ab5-a9b0-df184e552ce3 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:50:04.998595 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f58aa7-4c50-4e78-b3a5-2070ddfd196e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:50:04.999228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f58aa7-4c50-4e78-b3a5-2070ddfd196e admin admin] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:50:04.999500 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f58aa7-4c50-4e78-b3a5-2070ddfd196e admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:50:04.999768 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f58aa7-4c50-4e78-b3a5-2070ddfd196e admin admin] PATH_INFO: `/v3/projects` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:50:05.000440 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f58aa7-4c50-4e78-b3a5-2070ddfd196e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:50:05.002406 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f58aa7-4c50-4e78-b3a5-2070ddfd196e admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:50:05.103924 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1474/2961] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 31 11:50:04 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:50:05.109635 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb0645a-ef02-4cce-b0e8-14b5b3efc33a admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:50:05.121274 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb0645a-ef02-4cce-b0e8-14b5b3efc33a admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:50:05.192386 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a6f95-4ec8-4f31-83f8-a0fbbbcc02f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:50:05.193008 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a6f95-4ec8-4f31-83f8-a0fbbbcc02f9 admin admin] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:50:05.193286 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a6f95-4ec8-4f31-83f8-a0fbbbcc02f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:50:05.193560 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a6f95-4ec8-4f31-83f8-a0fbbbcc02f9 admin admin] PATH_INFO: `/v3/users` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:50:05.194222 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a6f95-4ec8-4f31-83f8-a0fbbbcc02f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:50:05.196317 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a6f95-4ec8-4f31-83f8-a0fbbbcc02f9 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:50:05.311458 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1488/2962] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 31 11:50:05 2022] POST /identity/v3/users => generated 287 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:50:05.316127 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f58aa7-4c50-4e78-b3a5-2070ddfd196e admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:50:05.328681 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f58aa7-4c50-4e78-b3a5-2070ddfd196e admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:50:05.334995 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefcbff8-c306-4877-8a8a-978f2306b9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:50:05.336186 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcbff8-c306-4877-8a8a-978f2306b9de admin admin] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:50:05.336685 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcbff8-c306-4877-8a8a-978f2306b9de admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:50:05.337163 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcbff8-c306-4877-8a8a-978f2306b9de admin admin] PATH_INFO: `/v3/roles` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:50:05.338359 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefcbff8-c306-4877-8a8a-978f2306b9de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:50:05.342858 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefcbff8-c306-4877-8a8a-978f2306b9de admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:50:05.362925 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1475/2963] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:50:05.368657 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a6f95-4ec8-4f31-83f8-a0fbbbcc02f9 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:50:05.387239 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a6f95-4ec8-4f31-83f8-a0fbbbcc02f9 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:50:05.393379 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff2f63-5adb-49a1-97bc-e97d7aae19e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:50:05.394156 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff2f63-5adb-49a1-97bc-e97d7aae19e3 admin admin] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:50:05.394366 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff2f63-5adb-49a1-97bc-e97d7aae19e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:50:05.394675 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff2f63-5adb-49a1-97bc-e97d7aae19e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:50:05.395338 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff2f63-5adb-49a1-97bc-e97d7aae19e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:50:05.396982 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff2f63-5adb-49a1-97bc-e97d7aae19e3 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:50:05.410419 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1489/2964] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 31 11:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:50:05.415133 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefcbff8-c306-4877-8a8a-978f2306b9de admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:50:05.426659 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefcbff8-c306-4877-8a8a-978f2306b9de admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:50:05.432095 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e674497-9ea0-4110-9987-3bddf6304f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:50:05.433026 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e674497-9ea0-4110-9987-3bddf6304f16 admin admin] REQUEST_METHOD: `PUT` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:50:05.433326 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e674497-9ea0-4110-9987-3bddf6304f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:50:05.433601 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e674497-9ea0-4110-9987-3bddf6304f16 admin admin] PATH_INFO: `/v3/projects/7416b58a2eb6404abf7a8724f964d3f5/users/27e654ce2609490c9123770ce1c2dcc5/roles/2278da3c52644a819d210b5f9c20b6c1` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:50:05.451784 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e674497-9ea0-4110-9987-3bddf6304f16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7416b58a2eb6404abf7a8724f964d3f5, user_id=27e654ce2609490c9123770ce1c2dcc5, role_id=2278da3c52644a819d210b5f9c20b6c1)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:50:05.455076 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e674497-9ea0-4110-9987-3bddf6304f16 admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:50:05.457939 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:50:05.550929 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1476/2965] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 31 11:50:05 2022] PUT /identity/v3/projects/7416b58a2eb6404abf7a8724f964d3f5/users/27e654ce2609490c9123770ce1c2dcc5/roles/2278da3c52644a819d210b5f9c20b6c1 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:50:05.565923 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ea502-ff9c-45fa-8812-c6aa7958a457 None None] REQUEST_METHOD: `POST` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:50:05.566369 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ea502-ff9c-45fa-8812-c6aa7958a457 None None] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:50:05.566902 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ea502-ff9c-45fa-8812-c6aa7958a457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:50:05.609545 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.auth.core [None req-420ea502-ff9c-45fa-8812-c6aa7958a457 None None] MFA Rules not processed for user `27e654ce2609490c9123770ce1c2dcc5`. Rule list: `[]` (Enabled: `True`). {{(pid=82088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:50:05.611122 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-420ea502-ff9c-45fa-8812-c6aa7958a457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:50:05.640588 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:50:05.693035 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1490/2966] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:50:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:50:05.702548 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e674497-9ea0-4110-9987-3bddf6304f16 admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:50:05.704453 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-1e674497-9ea0-4110-9987-3bddf6304f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:50:05.771876 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e674497-9ea0-4110-9987-3bddf6304f16 admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:50:05.775887 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c650ba-48e4-44da-a542-680934739208 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f9aeb9857b45d68c3baa8fc4acc5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:50:05.776553 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c650ba-48e4-44da-a542-680934739208 service cinder] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:50:05.776824 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c650ba-48e4-44da-a542-680934739208 service cinder] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:50:05.777127 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c650ba-48e4-44da-a542-680934739208 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:50:05.787345 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c650ba-48e4-44da-a542-680934739208 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:50:05.788902 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c650ba-48e4-44da-a542-680934739208 service cinder] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:50:05.800645 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1477/2967] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:50:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:50:15.703577 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=16 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:50:15.795524 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:50:20.404626 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ea502-ff9c-45fa-8812-c6aa7958a457 None None] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:50:20.484510 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ea502-ff9c-45fa-8812-c6aa7958a457 None None] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:50:20.489410 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6dc8d-088c-483f-915a-1529ba83443b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:50:20.490099 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6dc8d-088c-483f-915a-1529ba83443b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:50:20.490422 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6dc8d-088c-483f-915a-1529ba83443b admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:50:20.490724 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6dc8d-088c-483f-915a-1529ba83443b admin admin] PATH_INFO: `/v3/users/27e654ce2609490c9123770ce1c2dcc5` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:50:20.491999 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6dc8d-088c-483f-915a-1529ba83443b admin admin] RBAC: Authorizing `identity:delete_user(user_id=27e654ce2609490c9123770ce1c2dcc5)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:50:20.493994 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6dc8d-088c-483f-915a-1529ba83443b admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:50:20.712925 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-76d6dc8d-088c-483f-915a-1529ba83443b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27e654ce2609490c9123770ce1c2dcc5'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:50:20.729539 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-76d6dc8d-088c-483f-915a-1529ba83443b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27e654ce2609490c9123770ce1c2dcc5'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:50:20.737831 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-76d6dc8d-088c-483f-915a-1529ba83443b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27e654ce2609490c9123770ce1c2dcc5'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:50:20.821387 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-76d6dc8d-088c-483f-915a-1529ba83443b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27e654ce2609490c9123770ce1c2dcc5'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:50:20.822138 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-76d6dc8d-088c-483f-915a-1529ba83443b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27e654ce2609490c9123770ce1c2dcc5'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:50:20.829725 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1491/2968] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:50:20 2022] DELETE /identity/v3/users/27e654ce2609490c9123770ce1c2dcc5 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:50:20.834809 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b7228-4d99-44d1-88fc-90f689cb53b7 None None] REQUEST_METHOD: `POST` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:50:20.835429 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b7228-4d99-44d1-88fc-90f689cb53b7 None None] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:50:20.835583 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b7228-4d99-44d1-88fc-90f689cb53b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:50:20.858367 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.auth.core [None req-c39b7228-4d99-44d1-88fc-90f689cb53b7 None None] MFA Rules not processed for user `32a2719366d1460cb5f9b2c1c8e41774`. Rule list: `[]` (Enabled: `True`). {{(pid=82089) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:50:20.859842 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-c39b7228-4d99-44d1-88fc-90f689cb53b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:50:20.936766 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1478/2969] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 31 11:50:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:50:20.947294 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6dc8d-088c-483f-915a-1529ba83443b admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:50:20.948896 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-76d6dc8d-088c-483f-915a-1529ba83443b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:50:21.014978 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6dc8d-088c-483f-915a-1529ba83443b admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:50:21.018907 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcebfc8-dec8-46b3-9258-3740cdd4e9cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:50:21.019818 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcebfc8-dec8-46b3-9258-3740cdd4e9cb service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:50:21.020103 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcebfc8-dec8-46b3-9258-3740cdd4e9cb service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:50:21.020366 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcebfc8-dec8-46b3-9258-3740cdd4e9cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:50:21.031101 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcebfc8-dec8-46b3-9258-3740cdd4e9cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:50:21.032649 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcebfc8-dec8-46b3-9258-3740cdd4e9cb service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:50:21.043981 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1492/2970] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:50:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:50:21.544984 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b7228-4d99-44d1-88fc-90f689cb53b7 None None] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:50:21.546978 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-c39b7228-4d99-44d1-88fc-90f689cb53b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:50:21.567728 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b7228-4d99-44d1-88fc-90f689cb53b7 None None] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:50:21.578839 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4dba10-201c-4081-98e4-2cfb8936e53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:50:21.579692 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4dba10-201c-4081-98e4-2cfb8936e53a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:50:21.580127 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4dba10-201c-4081-98e4-2cfb8936e53a admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:50:21.580426 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4dba10-201c-4081-98e4-2cfb8936e53a admin admin] PATH_INFO: `/v3/projects/7416b58a2eb6404abf7a8724f964d3f5` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:50:21.581904 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4dba10-201c-4081-98e4-2cfb8936e53a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7416b58a2eb6404abf7a8724f964d3f5)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:50:21.584561 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4dba10-201c-4081-98e4-2cfb8936e53a admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:50:21.713678 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-af4dba10-201c-4081-98e4-2cfb8936e53a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7416b58a2eb6404abf7a8724f964d3f5', 'request_id': 'req-af4dba10-201c-4081-98e4-2cfb8936e53a'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:50:21.727337 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-af4dba10-201c-4081-98e4-2cfb8936e53a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7416b58a2eb6404abf7a8724f964d3f5', 'request_id': 'req-af4dba10-201c-4081-98e4-2cfb8936e53a'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:50:21.738143 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-af4dba10-201c-4081-98e4-2cfb8936e53a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7416b58a2eb6404abf7a8724f964d3f5', 'request_id': 'req-af4dba10-201c-4081-98e4-2cfb8936e53a'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:50:21.855563 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-af4dba10-201c-4081-98e4-2cfb8936e53a admin admin] The token cache is being invalidate because project 7416b58a2eb6404abf7a8724f964d3f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82089) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:50:21.857228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-af4dba10-201c-4081-98e4-2cfb8936e53a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:50:21.857569 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-af4dba10-201c-4081-98e4-2cfb8936e53a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:50:21.859295 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1479/2971] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:50:21 2022] DELETE /identity/v3/projects/7416b58a2eb6404abf7a8724f964d3f5 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:50:31.038770 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:50:31.851923 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:51:07.255348 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcebfc8-dec8-46b3-9258-3740cdd4e9cb service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:07.265027 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcebfc8-dec8-46b3-9258-3740cdd4e9cb service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:07.317116 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca24aa-fe39-4f55-9fa0-5507e187a0b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:07.317724 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca24aa-fe39-4f55-9fa0-5507e187a0b0 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:07.318001 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca24aa-fe39-4f55-9fa0-5507e187a0b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:07.318275 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca24aa-fe39-4f55-9fa0-5507e187a0b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:07.320060 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a6ca24aa-fe39-4f55-9fa0-5507e187a0b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:51:07.330483 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ca24aa-fe39-4f55-9fa0-5507e187a0b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:07.331924 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ca24aa-fe39-4f55-9fa0-5507e187a0b0 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:07.341097 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1493/2972] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:51:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:51:17.336470 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=26 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:51:37.204113 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4dba10-201c-4081-98e4-2cfb8936e53a admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:37.207637 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-af4dba10-201c-4081-98e4-2cfb8936e53a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:51:37.329383 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4dba10-201c-4081-98e4-2cfb8936e53a admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:37.338574 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea3f13b-fefe-4823-8ce5-8f2c3d27157c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11971a594014d86968c3356d11d42f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:37.339750 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea3f13b-fefe-4823-8ce5-8f2c3d27157c service nova] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:37.340281 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea3f13b-fefe-4823-8ce5-8f2c3d27157c service nova] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:37.340757 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea3f13b-fefe-4823-8ce5-8f2c3d27157c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:37.344113 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-eea3f13b-fefe-4823-8ce5-8f2c3d27157c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:51:37.372076 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea3f13b-fefe-4823-8ce5-8f2c3d27157c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:37.375356 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea3f13b-fefe-4823-8ce5-8f2c3d27157c service nova] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:37.398053 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1480/2973] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:51:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:51:37.741544 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca24aa-fe39-4f55-9fa0-5507e187a0b0 service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:37.744966 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-a6ca24aa-fe39-4f55-9fa0-5507e187a0b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:51:37.764861 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca24aa-fe39-4f55-9fa0-5507e187a0b0 service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:37.771975 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d23baeb-ca8a-4de6-bb8a-209db0a6eb83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:37.773125 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23baeb-ca8a-4de6-bb8a-209db0a6eb83 service neutron] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:37.773623 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23baeb-ca8a-4de6-bb8a-209db0a6eb83 service neutron] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:37.774121 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23baeb-ca8a-4de6-bb8a-209db0a6eb83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:37.777910 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-0d23baeb-ca8a-4de6-bb8a-209db0a6eb83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:51:37.807008 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d23baeb-ca8a-4de6-bb8a-209db0a6eb83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:37.809990 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d23baeb-ca8a-4de6-bb8a-209db0a6eb83 service neutron] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:37.832914 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1494/2974] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:51:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:51:47.388280 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=39 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:51:47.822489 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=6 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:51:52.279747 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea3f13b-fefe-4823-8ce5-8f2c3d27157c service nova] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:51:52.290732 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea3f13b-fefe-4823-8ce5-8f2c3d27157c service nova] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:51:52.294590 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d1f902-d150-470c-b3e8-6d7d47d88fdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b7f45b2e47b8a01048c36d00f742', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:51:52.295269 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d1f902-d150-470c-b3e8-6d7d47d88fdd service neutron] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:51:52.295493 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d1f902-d150-470c-b3e8-6d7d47d88fdd service neutron] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:51:52.295767 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d1f902-d150-470c-b3e8-6d7d47d88fdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:51:52.297631 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-35d1f902-d150-470c-b3e8-6d7d47d88fdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:51:52.384456 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d1f902-d150-470c-b3e8-6d7d47d88fdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:51:52.385799 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d1f902-d150-470c-b3e8-6d7d47d88fdd service neutron] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:51:52.395906 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1481/2975] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:51:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:52:00.338569 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d23baeb-ca8a-4de6-bb8a-209db0a6eb83 service neutron] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:00.344097 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-0d23baeb-ca8a-4de6-bb8a-209db0a6eb83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:00.493801 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d23baeb-ca8a-4de6-bb8a-209db0a6eb83 service neutron] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:00.506748 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec86258-eaff-45a3-a363-fc43ffc3963b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3db46a4b4e74cc0bb2a7bd6f0e99105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:00.506748 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec86258-eaff-45a3-a363-fc43ffc3963b service placement] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:00.506748 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec86258-eaff-45a3-a363-fc43ffc3963b service placement] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:00.506748 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec86258-eaff-45a3-a363-fc43ffc3963b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:00.507674 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-fec86258-eaff-45a3-a363-fc43ffc3963b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:00.508791 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d1f902-d150-470c-b3e8-6d7d47d88fdd service neutron] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:00.510449 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-35d1f902-d150-470c-b3e8-6d7d47d88fdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:00.525640 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec86258-eaff-45a3-a363-fc43ffc3963b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:00.526853 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d1f902-d150-470c-b3e8-6d7d47d88fdd service neutron] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:00.528153 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec86258-eaff-45a3-a363-fc43ffc3963b service placement] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:00.532611 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac352ecf-3987-4d7d-8800-cac6bb7c7f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:00.533362 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac352ecf-3987-4d7d-8800-cac6bb7c7f3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:00.538497 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac352ecf-3987-4d7d-8800-cac6bb7c7f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:00.538497 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac352ecf-3987-4d7d-8800-cac6bb7c7f3a admin admin] PATH_INFO: `/v3/users/8492a5ac560f43e4a3342f067dda8fef` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:00.538497 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac352ecf-3987-4d7d-8800-cac6bb7c7f3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8492a5ac560f43e4a3342f067dda8fef)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:00.538497 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac352ecf-3987-4d7d-8800-cac6bb7c7f3a admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:00.542883 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1495/2976] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:52:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:52:00.794877 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-ac352ecf-3987-4d7d-8800-cac6bb7c7f3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8492a5ac560f43e4a3342f067dda8fef'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:00.872383 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-ac352ecf-3987-4d7d-8800-cac6bb7c7f3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8492a5ac560f43e4a3342f067dda8fef'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:00.874113 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-ac352ecf-3987-4d7d-8800-cac6bb7c7f3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8492a5ac560f43e4a3342f067dda8fef'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:00.874113 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-ac352ecf-3987-4d7d-8800-cac6bb7c7f3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8492a5ac560f43e4a3342f067dda8fef'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:00.889300 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-ac352ecf-3987-4d7d-8800-cac6bb7c7f3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8492a5ac560f43e4a3342f067dda8fef'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:00.903058 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1482/2977] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:52:00 2022] DELETE /identity/v3/users/8492a5ac560f43e4a3342f067dda8fef => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:52:01.246085 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec86258-eaff-45a3-a363-fc43ffc3963b service placement] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:01.259898 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec86258-eaff-45a3-a363-fc43ffc3963b service placement] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:01.333440 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7d644-f9ad-49ff-98e5-fdc5e57ddc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:01.334074 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7d644-f9ad-49ff-98e5-fdc5e57ddc5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:01.334346 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7d644-f9ad-49ff-98e5-fdc5e57ddc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:01.334693 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7d644-f9ad-49ff-98e5-fdc5e57ddc5e admin admin] PATH_INFO: `/v3/projects/09c0f45a215341de9ab189e544a6ecd6` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:01.335876 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7d644-f9ad-49ff-98e5-fdc5e57ddc5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=09c0f45a215341de9ab189e544a6ecd6)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:01.338665 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7d644-f9ad-49ff-98e5-fdc5e57ddc5e admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:01.474883 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:52:01.476563 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-f3c7d644-f9ad-49ff-98e5-fdc5e57ddc5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09c0f45a215341de9ab189e544a6ecd6', 'request_id': 'req-f3c7d644-f9ad-49ff-98e5-fdc5e57ddc5e'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:01.546890 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-f3c7d644-f9ad-49ff-98e5-fdc5e57ddc5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09c0f45a215341de9ab189e544a6ecd6', 'request_id': 'req-f3c7d644-f9ad-49ff-98e5-fdc5e57ddc5e'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:01.553489 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-f3c7d644-f9ad-49ff-98e5-fdc5e57ddc5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09c0f45a215341de9ab189e544a6ecd6', 'request_id': 'req-f3c7d644-f9ad-49ff-98e5-fdc5e57ddc5e'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:01.569718 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-f3c7d644-f9ad-49ff-98e5-fdc5e57ddc5e admin admin] The token cache is being invalidate because project 09c0f45a215341de9ab189e544a6ecd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:52:01.570050 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-f3c7d644-f9ad-49ff-98e5-fdc5e57ddc5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:01.573609 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-f3c7d644-f9ad-49ff-98e5-fdc5e57ddc5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:01.577279 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1496/2978] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:52:01 2022] DELETE /identity/v3/projects/09c0f45a215341de9ab189e544a6ecd6 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:52:01.583639 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac352ecf-3987-4d7d-8800-cac6bb7c7f3a admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:01.585711 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-ac352ecf-3987-4d7d-8800-cac6bb7c7f3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:01.600780 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac352ecf-3987-4d7d-8800-cac6bb7c7f3a admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:01.658525 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:52:01.669228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265ca54-3317-4b52-8c57-68b3525c812f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:01.669867 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265ca54-3317-4b52-8c57-68b3525c812f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:01.670145 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265ca54-3317-4b52-8c57-68b3525c812f admin admin] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:01.670427 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265ca54-3317-4b52-8c57-68b3525c812f admin admin] PATH_INFO: `/v3/users/646f1e7724f14f29bf806bcc3a6a554c` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:01.671794 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265ca54-3317-4b52-8c57-68b3525c812f admin admin] RBAC: Authorizing `identity:delete_user(user_id=646f1e7724f14f29bf806bcc3a6a554c)` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:01.673771 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265ca54-3317-4b52-8c57-68b3525c812f admin admin] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:01.896821 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2265ca54-3317-4b52-8c57-68b3525c812f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '646f1e7724f14f29bf806bcc3a6a554c'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:02.044246 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2265ca54-3317-4b52-8c57-68b3525c812f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '646f1e7724f14f29bf806bcc3a6a554c'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:02.044868 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2265ca54-3317-4b52-8c57-68b3525c812f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '646f1e7724f14f29bf806bcc3a6a554c'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:02.045511 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2265ca54-3317-4b52-8c57-68b3525c812f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '646f1e7724f14f29bf806bcc3a6a554c'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:02.059374 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.notifications [None req-2265ca54-3317-4b52-8c57-68b3525c812f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '646f1e7724f14f29bf806bcc3a6a554c'} {{(pid=82089) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:02.072245 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1483/2979] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 31 11:52:01 2022] DELETE /identity/v3/users/646f1e7724f14f29bf806bcc3a6a554c => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:52:02.387420 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7d644-f9ad-49ff-98e5-fdc5e57ddc5e admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:02.389318 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-f3c7d644-f9ad-49ff-98e5-fdc5e57ddc5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:02.410647 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7d644-f9ad-49ff-98e5-fdc5e57ddc5e admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:02.497238 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d17437-50d2-490d-9734-df87ee4e6107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32a2719366d1460cb5f9b2c1c8e41774', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '776925ef060044f49d8dab7d8928f73c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:02.497797 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d17437-50d2-490d-9734-df87ee4e6107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:02.498081 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d17437-50d2-490d-9734-df87ee4e6107 admin admin] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:02.498378 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d17437-50d2-490d-9734-df87ee4e6107 admin admin] PATH_INFO: `/v3/projects/98ee52a6fe054db199b8f08bd5b6052e` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:02.499611 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d17437-50d2-490d-9734-df87ee4e6107 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98ee52a6fe054db199b8f08bd5b6052e)` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:02.501557 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d17437-50d2-490d-9734-df87ee4e6107 admin admin] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:02.613154 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-20d17437-50d2-490d-9734-df87ee4e6107 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98ee52a6fe054db199b8f08bd5b6052e', 'request_id': 'req-20d17437-50d2-490d-9734-df87ee4e6107'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:02.678100 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-20d17437-50d2-490d-9734-df87ee4e6107 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98ee52a6fe054db199b8f08bd5b6052e', 'request_id': 'req-20d17437-50d2-490d-9734-df87ee4e6107'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:02.685558 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-20d17437-50d2-490d-9734-df87ee4e6107 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98ee52a6fe054db199b8f08bd5b6052e', 'request_id': 'req-20d17437-50d2-490d-9734-df87ee4e6107'} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:02.699929 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-20d17437-50d2-490d-9734-df87ee4e6107 admin admin] The token cache is being invalidate because project 98ee52a6fe054db199b8f08bd5b6052e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:52:02.700218 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-20d17437-50d2-490d-9734-df87ee4e6107 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:02.700915 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.notifications [None req-20d17437-50d2-490d-9734-df87ee4e6107 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:52:02.702545 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1497/2980] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 31 11:52:02 2022] DELETE /identity/v3/projects/98ee52a6fe054db199b8f08bd5b6052e => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:52:06.669189 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265ca54-3317-4b52-8c57-68b3525c812f admin admin] Authenticating user token {{(pid=82089) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:52:06.670805 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-2265ca54-3317-4b52-8c57-68b3525c812f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:06.753510 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265ca54-3317-4b52-8c57-68b3525c812f admin admin] Validating token access rules against request {{(pid=82089) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:52:06.832193 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3d5fe-731e-4765-8f4c-82b2809ec74d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3db46a4b4e74cc0bb2a7bd6f0e99105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82089) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:52:06.832898 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3d5fe-731e-4765-8f4c-82b2809ec74d service placement] REQUEST_METHOD: `GET` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:52:06.833274 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3d5fe-731e-4765-8f4c-82b2809ec74d service placement] SCRIPT_NAME: `/identity` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:52:06.833590 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3d5fe-731e-4765-8f4c-82b2809ec74d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82089) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:52:06.835693 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.fernet_utils [None req-b0a3d5fe-731e-4765-8f4c-82b2809ec74d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82089) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:52:06.848913 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a3d5fe-731e-4765-8f4c-82b2809ec74d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:52:06.850877 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a3d5fe-731e-4765-8f4c-82b2809ec74d service placement] RBAC: Authorization granted {{(pid=82089) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:52:06.863228 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: [pid: 82089|app: 0|req: 1484/2981] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:52:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:52:12.695326 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=58,keystone:INSERT=2,keystone:DELETE=10 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:52:16.857521 dv-cinder-846173-5-iscsi devstack@keystone.service[82089]: DEBUG dbcounter [-] [82089] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82089) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:57:10.646764 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d17437-50d2-490d-9734-df87ee4e6107 admin admin] Authenticating user token {{(pid=82088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:57:10.648215 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.fernet_utils [None req-20d17437-50d2-490d-9734-df87ee4e6107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:57:10.689558 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d17437-50d2-490d-9734-df87ee4e6107 admin admin] Validating token access rules against request {{(pid=82088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:57:10.694959 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0eca3-2b9f-48cf-a311-fc3b5d4008d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3db46a4b4e74cc0bb2a7bd6f0e99105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db9b3f8a9f041938ac7b39003a35015', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:57:10.695577 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0eca3-2b9f-48cf-a311-fc3b5d4008d6 service placement] REQUEST_METHOD: `GET` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:57:10.695863 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0eca3-2b9f-48cf-a311-fc3b5d4008d6 service placement] SCRIPT_NAME: `/identity` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:57:10.696136 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0eca3-2b9f-48cf-a311-fc3b5d4008d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:57:10.706704 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af0eca3-2b9f-48cf-a311-fc3b5d4008d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:57:10.708343 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af0eca3-2b9f-48cf-a311-fc3b5d4008d6 service placement] RBAC: Authorization granted {{(pid=82088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:57:10.720494 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: [pid: 82088|app: 0|req: 1498/2982] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 31 11:57:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:57:20.715522 dv-cinder-846173-5-iscsi devstack@keystone.service[82088]: DEBUG dbcounter [-] [82088] Writing DB stats keystone:SELECT=17 {{(pid=82088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}